Adult education directors' perceptions of performance indicators for adult education programs in the state of Florida

Current demands for accountability in education emphasize outcome-based program evaluation and tie program funding to individual student performance. As has been the case for elementary and secondary programs, demands for accountability have increased pressure on adult educators to show evidence of the benefits of their programs in order to justify their financial support. In Florida, recent legislation fundamentally changes the delivery of adult education in the state by establishing a performance-based funding system that is based on outcomes related to the retention, completion, and employment of program participants.^ A performance-based funding system requires an evaluation process that stresses outcome indicators over indicators that focus on program context or process. Although the state has adopted indicators of program quality to evaluate its adult education programs, these indicators focus mostly on program processes rather than student outcomes. In addition, the indicators are not specifically tied to workforce development outcomes, a priority to federal and local funding agents.^ Improving the accountability of adult education programs and defining the role of these programs in Florida's Workforce Development System has become a priority to policy makers across the state. Another priority has been to involve adult education practitioners in every step of this process.^ This study was conducted in order to determine what performance indicators, as judged by the directors and supervisors of adult education programs in the state of Florida, are important and feasible in measuring the quality and effectiveness of these programs. The results of the study indicated that, both statewide and by region, the respondents consistently gave the highest ratings on both importance and feasibility to the indicators of Program Context, which reflect the needs, composition, and structure of the programs, and to the indicators of Educational Gain, which reflect learner progress in the attainment of basic skills and competencies. In turn, the respondents gave the lowest ratings on both importance and feasibility to the indicators in the areas of Return on State's Investment, Efficiency, Retention, and Workforce Training. In general, the indicators that received high ratings for importance also received high ratings for feasibility. ^

Community college faculty commitment to Florida performance -based funding indicators for academic programs

In 1996, the State of Florida implemented a performance-based funding program for the Associate in Arts degree offered by community colleges. Additional funds are allocated for distribution among public community colleges based on performance indicators. The indicators are comprised of 10 performance goals that refer to productivity indexed by overall degree completions as well as subgroups: special disadvantaged populations, transfers, job placements, and education acceleration. ^ This study examined the level of self-reported commitment of community college faculty to the 10 Florida performance-based funding indicators for academic programs. Also examined were the relationships between commitment and (a) self-efficacy in contributing to the achievement of the indicators and (b) personal financial reward expectation for contributing to the achievement of the indicators. The relationships between commitment and (a) gender, (b) academic rank, and (c) types of courses taught were analyzed based on secondary analyses. ^ The participants were 303 full-time faculty members of Miami-Dade Community College who taught courses taken by students pursuing the Associate in Arts degree. A questionnaire was developed to measure commitment, self-efficacy, and expectation of financial reward for each of the 10 indicators. ^ The mean composite commitment score for faculty members who responded to the survey was 4.07 in a scale of 1 to 5. Greater commitment was reported for indicators closely related to the traditional mission of community colleges (i.e., facilitating progress of special groups in earning the AA degree in preparation for transferring to a four-year university). Lower commitment was reported for indicators oriented to State priorities such as education acceleration mechanisms and job placements. Commitment was correlated with three variables: self-efficacy, expectation of financial reward, and types of courses taught. However, commitment was not related to gender and academic rank. Although a cause-effect relationship cannot be inferred from this study, the findings depict a positive relationship between faculty commitment to performance-based funding indicators and faculty self-efficacy to contribute to the achievement of the indicators. ^

A methodological framework for modeling pavement maintenance costs for projects with performance-based contracts

Performance-based maintenance contracts differ significantly from material and method-based contracts that have been traditionally used to maintain roads. Road agencies around the world have moved towards a performance-based contract approach because it offers several advantages like cost saving, better budgeting certainty, better customer satisfaction with better road services and conditions. Payments for the maintenance of road are explicitly linked to the contractor successfully meeting certain clearly defined minimum performance indicators in these contracts. Quantitative evaluation of the cost of performance-based contracts has several difficulties due to the complexity of the pavement deterioration process. Based on a probabilistic analysis of failures of achieving multiple performance criteria over the length of the contract period, an effort has been made to develop a model that is capable of estimating the cost of these performance-based contracts. One of the essential functions of such model is to predict performance of the pavement as accurately as possible. Prediction of future degradation of pavement is done using Markov Chain Process, which requires estimating transition probabilities from previous deterioration rate for similar pavements. Transition probabilities were derived using historical pavement condition rating data, both for predicting pavement deterioration when there is no maintenance, and for predicting pavement improvement when maintenance activities are performed. A methodological framework has been developed to estimate the cost of maintaining road based on multiple performance criteria such as crack, rut and, roughness. The application of the developed model has been demonstrated via a real case study of Miami Dade Expressways (MDX) using pavement condition rating data from Florida Department of Transportation (FDOT) for a typical performance-based asphalt pavement maintenance contract. Results indicated that the pavement performance model developed could predict the pavement deterioration quite accurately. Sensitivity analysis performed shows that the model is very responsive to even slight changes in pavement deterioration rate and performance constraints. It is expected that the use of this model will assist the highway agencies and contractors in arriving at a fair contract value for executing long term performance-based pavement maintenance works.

Acceptable vs. marginal police officers' psychological ratings : a longitudinal comparison of job performance

Archival research was conducted on the inception of preemployment psychological testing, as part of the background screening process, to select police officers for a local police department. Various issues and incidents were analyzed to help explain why this police department progressed from an abbreviated version of a psychological battery, to a much more sophisticated and comprehensive set of instruments. While doubts about psychological exams do exist, research has shown that many are valid and reliable in predicting job performance of police candidates. During a three year period, a police department hired 162 candidates (133 males and 29 females) who received "acceptable" psychological ratings and 71 candidates (58 males and 13 females) who received "marginal" psychological ratings. A document analysis consisted of variables that have been identified as job performance indicators which police psychological testing tries to predict, and "screen in" or "screen out" appropriate applicants. The areas of focus comprised the 6-month police academy, the 4-month Field Training Officer (FTO) Program, the remaining probationary period, and yearly performance up to five years of employment. Specific job performance variables were the final academy grade average, supervisors' evaluation ratings, reprimands, commendations, awards, citizen complaints, time losses, sick time usage, reassignments, promotions, and separations. A causal-comparative research design was used to determine if there were significant statistical differences in these job performance variables between police officers with "acceptable" psychological ratings and police officers with "marginal" psychological ratings. The results of multivariate analyses of variance, t-tests, and chi-square procedures as applicable, showed no significant differences between the two groups on any of the job performance variables.

A Methodological Framework for Modeling Pavement Maintenance Costs for Projects with Performance-based Contracts

Performance-based maintenance contracts differ significantly from material and method-based contracts that have been traditionally used to maintain roads. Road agencies around the world have moved towards a performance-based contract approach because it offers several advantages like cost saving, better budgeting certainty, better customer satisfaction with better road services and conditions. Payments for the maintenance of road are explicitly linked to the contractor successfully meeting certain clearly defined minimum performance indicators in these contracts. Quantitative evaluation of the cost of performance-based contracts has several difficulties due to the complexity of the pavement deterioration process. Based on a probabilistic analysis of failures of achieving multiple performance criteria over the length of the contract period, an effort has been made to develop a model that is capable of estimating the cost of these performance-based contracts. One of the essential functions of such model is to predict performance of the pavement as accurately as possible. Prediction of future degradation of pavement is done using Markov Chain Process, which requires estimating transition probabilities from previous deterioration rate for similar pavements. Transition probabilities were derived using historical pavement condition rating data, both for predicting pavement deterioration when there is no maintenance, and for predicting pavement improvement when maintenance activities are performed. A methodological framework has been developed to estimate the cost of maintaining road based on multiple performance criteria such as crack, rut and, roughness. The application of the developed model has been demonstrated via a real case study of Miami Dade Expressways (MDX) using pavement condition rating data from Florida Department of Transportation (FDOT) for a typical performance-based asphalt pavement maintenance contract. Results indicated that the pavement performance model developed could predict the pavement deterioration quite accurately. Sensitivity analysis performed shows that the model is very responsive to even slight changes in pavement deterioration rate and performance constraints. It is expected that the use of this model will assist the highway agencies and contractors in arriving at a fair contract value for executing long term performance-based pavement maintenance works.

Parental involvement: Its relation to achievement in a multiethnic sample of newly immigrated students

The current study examined how parental involvement is related to the academic adaptation and performance of a multiethnic sample of newly immigrant students from South America and the Caribbean---Argentina, Colombia, Cuba, Haiti, and English-speaking Caribbean countries. Six hundred twenty-two participants attending elementary, middle and high school in Miami-Dade and Broward Counties were interviewed during two consecutive years at their respective schools. Parents completed surveys at home and returned them by mail. ^ Three forms of parent involvement were chosen as predictor variables---(1) parental contact and communication with school, (2) parental monitoring of children's school-related activities at home, and (3) parental expectations regarding children's academic achievement. Academic performance indicators obtained from school records were students' grade point averages for math and reading (GPA), FCAT-SSS (Florida Comprehensive Assessment Test-Sunshine State Standards) scores, and FCAT-NRT (Norm Referenced Test) scores. Hypotheses were (a) that the forms of parent involvement would differ across the different cultural groups and (b) that parent involvement would relate to students' achievement, regardless of the specific form of involvement. ^ Results indicated that Cuban and Colombian parents differed significantly in terms of monitoring school-related activities of their children and had higher expectations for their children than Haitian, West-Indian, or Argentinean parents. Cuban parents were more likely to be actively engaged in communication with their children's schools than parents of the other ethnic groups. Multiple regression analyses indicated that the three forms of parent involvement accounted for a significant proportion of GPA and FCAT-SSS scores, after controlling for the effects of demographic variables---grade level, gender, and ethnicity. Given interactions of the parent involvement variables by country of origin for GPA, separate analyses indicated that parent involvement appeared to be most effective for Cubans and West Indians. Results are discussed in terms of the need for experimental intervention studies to identify causal relationships between parent involvement and academic achievement. ^

Deprofessionalization of curriculum decision -making

The purpose of this study was to analyze the evolution of Florida state level policy efforts and to assess the responding educational policy development and implementation at the local school district level. The focus of this study was the secondary language arts curriculum in Miami-Dade County Public Schools. ^ Data was collected using document analysis as a source of meaning making out of the language sets proffered by agencies at each level. A matrix was created based on Klein's levels of curriculum decision-making and Functional Process Theory categories of policy formation. The matrix allowed the researcher to code and classify specific information in terms accountability/high-stakes testing; authority; outside influences; and operational/structural organization. ^ Federal policy documents provided a background and impetus for much of what originated at the State level. The State then produced policy directives which were accepted by the District and specific policy directives and guidelines for practice. No evidence was found indicating the involvement of any other agencies in the development, transmission or implementation of the State level initiated policies. ^ After analyzing the evolutionary process, it became clear that state policy directives were never challenged or discussed. Rather, they were accepted as standards to be met and as such, school districts complied. Policy implementation is shown to be a top-down phenomenon. No evidence was found indicating a dialogue between state and local systems, rather the state, as the source of authority, issued specifically worded policy directives and the district complied. Finally, this study recognizes that outside influences play an important role in shaping the education reform policy in the state of Florida. The federal government, through NCLB and other initiatives created a climate which led almost naturally to the creation of the Florida A+ Plan. Similarly, the concern of the business community, always interested in the production of competent workers, continued to support efforts at raising the minimum skill level of Florida high school graduates. ^ Suggestions are made for future research including the examination of local school sites in order to assess the overall nature of the school experience rather than rely upon performance indicators mandated by state policy. ^

Deprofessionalization of curriculum decision-making

The purpose of this study was to analyze the evolution of Florida state level policy efforts and to assess the responding educational policy development and implementation at the local school district level. The focus of this study was the secondary language arts curriculum in Miami-Dade County Public Schools. Data was collected using document analysis as a source of meaning making out of the language sets proffered by agencies at each level. A matrix was created based on Klein's levels of curriculum decision-making and Functional Process Theory categories of policy formation. The matrix allowed the researcher to code and classify specific information in terms accountability/high-stakes testing; authority; outside influences; and operational/structural organization. Federal policy documents provided a background and impetus for much of what originated at the State level. The State then produced policy directives which were accepted by the District and specific policy directives and guidelines for practice. No evidence was found indicating the involvement of any other agencies in the development, transmission or implementation of the State level initiated policies. After analyzing the evolutionary process, it became clear that state policy directives were never challenged or discussed. Rather, they were accepted as standards to be met and as such, school districts complied. Policy implementation is shown to be a top-down phenomenon. No evidence was found indicating a dialogue between state and local systems, rather the state, as the source of authority, issued specifically worded policy directives and the district complied. Finally, this study recognizes that outside influences play an important role in shaping the education reform policy in the state of Florida. The federal government, through NCLB and other initiatives created a climate which led almost naturally to the creation of the Florida A+ Plan. Similarly, the concern of the business community, always interested in the production of competent workers, continued to support efforts at raising the minimum skill level of Florida high school graduates. Suggestions are made for future research including the examination of local school sites in order to assess the overall nature of the school experience rather than rely upon performance indicators mandated by state policy.

Multinationality and performance: The role of cultural dissimilarity

The purpose of this research was to examine the influence of cultural dissimilarity on the relationship between multinationality and performance. Both direct and indirect effects were studied. In addition, the form of the multinationality-performance relationship was investigated.^ Five indicators of cultural dissimilarity were developed on the basis of Hofstede's cultural dimensions. Performance was measured along two dimensions--financial and operational. Multinationality was operationalized as the ratio of foreign sales to total sales. Secondary data was used for all variables in the study. The sample of firms comprised multinationals based in the United States from four global industries--chemicals, computers and office equipment, electrical and electrical goods, and drugs and pharmaceuticals.^ Regression analyses using pooled cross-section/time-series data indicated that the relationship between multinationality and performance is curvilinear. No direct effects of cultural dissimilarity on performance were found. However, the results show a moderating effect of cultural dissimilarity on the multinationality-performance relationship. The direction of this effect was positive for four of the five cultural dissimilarity measures. ^

The relation of sports participation to academic performance of high school students

Over the past one hundred years, interscholastic athletic programs have evolved to a place of prominence in both public and private education across America. The National Federation of State High School Associations (NFHS) estimates that approximately 3.96 million males and 2.80 million females participated in organized high school athletic programs during the 2001–2002 school year at over 17 thousand public and private high schools. The popularity of interscholastic athletic programs has resulted in continuous investigations of the relationship between high school athletic programs and academic performance. ^ The present study extends earlier investigations by examining the relation of athletic participation to several indicators of academic performance for senior high school students. This research examined: (a) average daily attendance of varsity athletes and non-athletes; (b) final cumulative grade point average; and (c) test scores on the tenth grade Florida Comprehensive Assessment Test (FLAT) in both reading and in mathematics. ^ Data were collected on 2081 randomly selected male and female high school students identified as athletes or non-athletes at ten public senior high schools in the Miami-Dade County Public Schools district. The results of the overall analyses showed a positive and significant relationship between athletic participation and educational performance. On average, athletes were absent fewer days from school per year than non-athletes and athletes earned a significantly higher cumulative grade point average than their non-athlete peers. A significant statistical difference was also found in the tenth grade FCAT test scores in both reading and mathematics for athletes and non-athletes when eighth grade FCAT test scores in reading and mathematics were used as co-variates. Athletes earned significantly higher Grade 10 FCAT test scores in both reading and mathematics than non-athletes. ^ Although cause and effect cannot be inferred from this study, the findings do indicate the potentially beneficial value of athletic programs in public secondary education. The study concluded that Florida high school graduation requirements might seriously consider the role of interscholastic athletic programs as a valid and essential extra-curricular activity. ^

Invasive exotic plant indicators for ecosystem restoration: An example from the Everglades restoration program

We have developed a comprehensive ecological indicator for invasive exotic plants, a human-influenced component of the Everglades that could threaten the success of the restoration initiative. Following development of a conceptual ecological model for invasive exotic species, presented as a companion paper in this special issue, we developed criteria to evaluate existing invasive exotic monitoring programs for use in developing invasive exotic performance measures. We then used data from the selected monitoring programs to define specific performance measures, using species presence and abundance as the basis of the indicator for invasive exotic plants. We then developed a series of questions used to evaluate region and/or individual species status with respect to invasion. Finally, we used an expert panel who had answered the questions for invasive exotic plants in the Everglades Lake Okeechobee model to develop a stoplight restoration report card to communicate invasive exotic plant status. The report card system provides a way to effectively evaluate and present indicator data to managers, policy makers, and the public using a uniform format among indicators. Collectively, the model, monitoring assessment, performance measures, and report card enable us to evaluate how invasive plants are impacting the restoration program and how effectively that impact is being managed. Applied through time, our approach also allows us to follow the progress of management actions to control the spread and reduce the impacts of invasive species and can be easily applied and adapted to other large-scale ecosystem projects.

Ecological indicators for system-wide assessment of the greater everglades ecosystem restoration program

Developing scientifically credible tools for measuring the success of ecological restoration projects is a difficult and a non-trivial task. Yet, reliable measures of the general health and ecological integrity of ecosystems are critical for assessing the success of restoration programs. The South Florida Ecosystem Restoration Task Force (Task Force), which helps coordinate a multi-billion dollar multi-organizational effort between federal, state, local and tribal governments to restore the Florida Everglades, is using a small set of system-wide ecological indicators to assess the restoration efforts. A team of scientists and managers identified eleven ecological indicators from a field of several hundred through a selection process using 12 criteria to determine their applicability as part of a system-wide suite. The 12 criteria are: (1) is the indicator relevant to the ecosystem? (2) Does it respond to variability at a scale that makes it applicable to the entire system? (3) Is the indicator feasible to implement and is it measureable? (4) Is the indicator sensitive to system drivers and is it predictable? (5) Is the indicator interpretable in a common language? (6) Are there situations where an optimistic trend with regard to an indicator might suggest a pessimistic restoration trend? (7) Are there situations where a pessimistic trend with regard to an indicator may be unrelated to restoration activities? (8) Is the indicator scientifically defensible? (9) Can clear, measureable targets be established for the indicator to allow for assessments of success? (10) Does the indicator have specificity to be able to result in corrective action? (11) What level of ecosystem process or structure does the indicator address? (12) Does the indicator provide early warning signs of ecological change? In addition, a two page stoplight report card was developed to assist in communicating the complex science inherent in ecological indicators in a common language for resource managers, policy makers and the public. The report card employs a universally understood stoplight symbol that uses green to indicate that targets are being met, yellow to indicate that targets have not been met and corrective action may be needed and red to represent that targets are far from being met and corrective action is required. This paper presents the scientific process and the results of the development and selection of the criteria, the indicators and the stoplight report card format and content. The detailed process and results for the individual indicators are presented in companion papers in this special issue of Ecological Indicators.

Submerged aquatic vegetation and bulrush in Lake Okeechobee as indicators of greater Everglades ecosystem restoration

Lake Okeechobee, Florida, located in the middle of the larger Kissimmee River-Lake Okeechobee-Everglades ecosystem in South Florida, serves a variety of ecosystem and water management functions including fish and wildlife habitat, flood control, water supply, and source water for environmental restoration. As a result, the ecological status of Lake Okeechobee plays a significant role in defining the overall success of the greater Everglades ecosystem restoration initiative. One of the major ecological indicators of Lake Okeechobee condition focuses on the near-shore and littoral zone regions as characterized by the distribution and abundance of submerged aquatic vegetation (SAV) and giant bulrush (Scirpus californicus(C.A. Mey.) Steud.). The objective of this study is to present a stoplight restoration report card communication system, common to all 11 indicators noted in this special journal issue, as a means to convey the status of SAV and bulrush in Lake Okeechobee. The report card could be used by managers, policy makers, scientists and the public to effectively evaluate and distill information about the ecological status in South Florida. Our assessment of the areal distribution of SAV in Lake Okeechobee is based on a combination of empirical SAV monitoring and output from a SAV habitat suitability model. Bulrush status in the lake is related to a suitability index linked to adult survival and seedling establishment metrics. Overall, presentation of these performance metrics in a stoplight format enables an evaluation of how the status of two major components of Lake Okeechobee relates to the South Florida restoration program, and how the status of the lake influences restoration efforts in South Florida.

Alligators and crocodiles as indicators for restoration of Everglades ecosystems

Alligators and crocodiles integrate biological impacts of hydrological operations, affecting them at all life stages through three key aspects of Everglades ecology: (1) food webs, (2) diversity and productivity, and (3) freshwater flow. Responses of crocodilians are directly related to suitability of environmental conditions and hydrologic change. Correlations between biological responses and environmental conditions contribute to an understanding of species’ status and trends over time. Positive or negative trends of crocodilian populations relative to hydrologic changes permit assessment of positive or negative trends in restoration. The crocodilian indicator uses monitoring parameters (performance measures) that have been shown to be both effective and efficient in tracking trends. The alligator component uses relative density (reported as an encounter rate), body condition, and occupancy rates of alligator holes; the crocodile component uses juvenile growth and hatchling survival. We hypothesize that these parameters are correlated with hydrologic conditions including depth, duration, timing, spatial extent and water quality. Salinity is a critical parameter in estuarine habitats. Assessments of parameters defined for crocodilian performance measures support these hypotheses. Alligators and crocodiles are the charismatic megafauna of the Everglades. They are both keystone and flagship species to which the public can relate. In addition, the parameters used to track trends are easy to understand. They provide answers to the following questions: How has the number of alligators or crocodiles changed? Are the animals fatter or thinner than they should be? Are the animals in the places (in terms of habitat and geography) where they should be? As surely as there is no other Everglades, no other single species defines the Everglades as does the American alligator. The Everglades is the only place in the world where both alligators and crocodiles exist. Crocodilians clearly respond to changes in hydrologic parameters of management interest. These relationships are easy to communicate and mean something to managers, decision makers, and the public. Having crocodilians on the list of system-wide, general indicators provides us with one of the most powerful tools we have to communicate progress of ecosystem restoration in Greater Everglades ecosystems to diverse audiences.