European university-based study programmes in evaluation: characteristics and future challenges

This chapter examines the state of evaluation training programs at European universities in 2012. It summarises the results of a survey that was conducted among representatives of 15 programs located in Belgium, Denmark, Greece, Italy, France, The Netherlands, Romania, Spain, Sweden and Switzerland. Some basic information about the programs are reported (e.g. organising body, degree offered, admission requirements, duration in months, price), as well as the programs’ core subjects and learning outcomes. The chapter discusses the challenges for university-based study programmes that arise from the current situation of the evaluation profession, and concludes with some thoughts on education and training as requirements for professionalisation in evaluation

A MANAGEMENT INFORMATION SYSTEM AND EVALUATION SCHEME FOR THE UNIVERSITY OF TEXAS EMPLOYEE ASSISTANCE PROGRAMS

Early Employee Assistance Programs (EAPs) had their origin in humanitarian motives, and there was little concern for their cost/benefit ratios; however, as some programs began accumulating data and analyzing it over time, even with single variables such as absenteeism, it became apparent that the humanitarian reasons for a program could be reinforced by cost savings particularly when the existence of the program was subject to justification.^ Today there is general agreement that cost/benefit analyses of EAPs are desirable, but the specific models for such analyses, particularly those making use of sophisticated but simple computer based data management systems, are few.^ The purpose of this research and development project was to develop a method, a design, and a prototype for gathering managing and presenting information about EAPS. This scheme provides information retrieval and analyses relevant to such aspects of EAP operations as: (1) EAP personnel activities, (2) Supervisory training effectiveness, (3) Client population demographics, (4) Assessment and Referral Effectiveness, (5) Treatment network efficacy, (6) Economic worth of the EAP.^ This scheme has been implemented and made operational at The University of Texas Employee Assistance Programs for more than three years.^ Application of the scheme in the various programs has defined certain variables which remained necessary in all programs. Depending on the degree of aggressiveness for data acquisition maintained by program personnel, other program specific variables are also defined. ^

Development and formative evaluation of an obesity prevention intervention for the pediatric primary care setting: Helping HAND (Healthy Activity and Nutrition Directions)

As the obesity epidemic continues to increase, the pediatric primary care office setting remains a relatively unexplored arena to offer obesity prevention interventions for children. The increased risk for adult obesity among 10 to 14 year-old children who are overweight, suggests obesity prevention programs should be introduced just before this age or early in this age period. Research is also accumulating on the importance of targeting parents along with children, since parents are in charge of the home environment for children. Therefore, the aim of this project was to develop an obesity prevention program called Helping HAND (Healthy Activity and Nutrition Directions) based on Social Cognitive Theory and authoritative parenting techniques for the pediatric primary care setting and conduct one-on-one interviews with parents as the initial formative evaluation of the intervention material for the obesity prevention intervention. A secondary aim of the project was to determine the feasibility of identifying appropriate subjects for the intervention, and conducting qualitative evaluations of the materials through recruitment through pediatric primary care settings. ^

Get ur 60: A social marketing campaign to promote and facilitate physical activity among Central Texas middle school students

If allowed to continue unabated, the obesity epidemic may lead to the first decline in life expectancy in the developed world (Olshansky et al., 2005). Similar to the relationship between smoking habits in youth and adulthood, obesogenic dietary and physical activity habits in childhood may persist into adulthood (Kelder et al., 2002). Teaching children how to establish healthy eating habits and activity levels, as well as providing them the necessary resources to internalize and maintain these behaviors, may be the key to curbing this epidemic.^ A school-based obesity prevention approach is advantageous for many reasons including exposure to large captive audiences, reduced costs of sustainability and long-term maintenance, and generalizability of models and results across multiple populations. The effectiveness of school-based programs has been researched over the past 20 years, with promising results.^ Social marketing is a program-planning process that “facilitates the acceptance, rejection, modification, abandonment, or maintenance of particular behaviors” (Grier & Bryant, 2005). Social marketing has been shown to be effective in a variety of public health applications including improving diet, increasing physical activity, and preventing substance abuse. It is hypothesized that social marketing could further enhance the effectiveness of the Coordinated Approach To Child Health (CATCH) Central Texas Middle School Project, a school-based obesity prevention program.^ The development, implementation, and initial evaluation of the get ur 60 campaign, to promote the Center for Disease Control and Prevention (CDC) recommended sixty minutes of daily activity, is described in this paper. Various components of the get ur 60 campaign were assessed to evaluate the effectiveness of the campaign during the first semester of implementation. At the end of the spring semester focus groups were held to collect student reactions to the first semester of the get ur 60 campaign.^ The initial results from the first semester of get ur 60 have demonstrated that the campaign as designed was feasible to implement, accepted at all intervention schools, and resulted in a measure of success. ^

An evaluation of Employee Assistance Programs: An overview and case study

Methodologies for the analysis and cross-evaluations of Employee Assistance Programs (EAPs) are necessary to indicate program weaknesses and strengths for both present and future EAP services.^ As EAPs have matured from simplistic industrial alcoholism programs to all problem-encompassing, complex programs, an aggressive group of entrepreneurs has developed. Business and industrial managements need established criteria to select the most suitable professional services for their needs, as well as the best investment value for cost containments.^ Four major EAPs were compared to determine critical areas necessary for cross evaluation. Future standardization of terminology and definitions of services will facilitate both the evaluation of a specific program and its comparison to problems or successes of other EAPs.^ Past evaluation difficulties have been caused by the lack of EAP program accreditations and professional personnel licensure requirements. The Association of Labor Management Administrators and Consultants on Alcoholism (ALMACA) has recently set certification criteria for personnel, which will enhance the professional standards for EAPs. ^

A DESCRIPTION AND EVALUATION OF EDUCATIONAL PROGRAMS FOR HEALTH MANPOWER IN NIGERIA

A descriptive study of the current educational programs of selected health personnel in Nigeria was made in 1986. Data on the content of educational programs was obtained from personal communication with the Heads of the various institutions and from their published materials (catalogs, course outlines and program descriptions). Adequacy of these programs was judged in the light of current health problems and needs of the population. Evaluation was based on the following criteria: (a) Selection of students to maximize their usefulness in the provision of health care. (b) Relevance of the curriculum to the tasks the trainee will be called upon to perform. (c) Types of courses that focus on community health needs. Using official reports, the health situation in the country was described to give a relative priority of health services.^ Findings indicate the following: (1) Health conditions in Nigeria are related to a high prevalence of illness and disease, unsanitary living conditions, a high ratio of infant mortality and a shortage of public health services. Priority needs for improvement call for attitudinal and environmental changes. (2) All health training programs have improved the relevance of education to community health needs by strengthening practical field experience, and teaching those courses which focus on disease prevention. (3) Prospective nurses and community health workers are selected on the basis of a number of personal and intellectual characteristics, but academic performance alone is the criterion for medical students. (4) The curriculum in the medical school needs to be restructured to cut back on time devoted to enriching the medical "background". Basic sciences need better integration with hospital work. (5) Managerial and organization courses have been well incorporated into the nursing and community health workers' curricula. (6) There is a marked overlap in the tasks the community health workers are expected to perform. This causes some redundancy in having four separate categories of these health personnel. ^

Effects of a health advocacy program to promote physical activity and nutrition behaviors among adolescents: An evaluation of the Young Leaders for Healthy Change program

Background. This study was designed to evaluate the effects of the Young Leaders for Healthy Change program, an internet-delivered program in the school setting that emphasized health advocacy skills-development, on nutrition and physical activity behaviors among older adolescents (13–18 years). The program consisted of online curricular modules, training modules, social media, peer and parental support, and a community service project. Module content was developed based on Social Cognitive Theory and known determinants of behavior for older adolescents. ^ Methods. Of the 283 students who participated in the fall 2011 YL program, 38 students participated in at least ten of the 12 weeks and were eligible for this study. This study used a single group-only pretest/posttest evaluation design. Participants were 68% female, 58% white/Caucasian, 74% 10th or 11th graders, and 89% mostly A and/or B students. The primary behavioral outcomes for this analysis were participation in 60-minutes of physical activity per day, 20-minutes of vigorous- or moderate- intensity physical activity (MVPA) participation per day, television and computer time, fruit and vegetable (FV) intake, sugar-sweetened beverage intake, and consumption of breakfast, home-cooked meals, and fast food. Other outcomes included knowledge, beliefs, and attitudes related to healthy eating, physical activity, and advocacy skills. ^ Findings. Among the 38 participants, no significant changes in any variables were observed. However, among those who did not previously meet behavioral goals there was an 89% increase in students who participated in more than 20 minutes of MVPA per day and a 58% increase in students who ate home-cooked meals 5–7 days per week. The majority of participants met program goals related to knowledge, beliefs, and attitudes prior to the start of the program. Participants reported either maintaining or improving to the goal at posttest for all items except FV intake knowledge, taste and affordability of healthy foods, interest in teaching others about being healthy, and ease of finding ways to advocate in the community. ^ Conclusions. The results of this evaluation indicated that promoting healthy behaviors requires different strategies than maintaining healthy behaviors among high school students. In the school setting, programs need to target the promotion and maintenance of health behaviors to engage all students who participate in the program as part of a class or club activity. Tailoring the program using screening and modifying strategies to meet the needs of all students may increase the potential reach of the program. The Transtheoretical Model may provide information on how to develop a tailored program. Additional research on how to utilize the constructs of TTM effectively among high school students needs to be conducted. Further evaluation studies should employ a more expansive evaluation to assess the long-term effectiveness of health advocacy programming.^

Social Workers' Perceptions of Family Preservation Programs

The passage of the Adoptions and Safe Families Act of 1997, with its focus on child safety and concurrent planning, has presented family preservation workers with new challenges and new opportunities. Twenty volunteers from a large comprehensive social service agency were interviewed to determine their experiences with two models of family preservation—Multisystemic Therapy (MST) and Traditional Family Preservation Service (TFPS) or practice as usual. Workers from both programs were able to articulate values consistent with family preservation as important strengths of the programs— keeping families together and empowering families for example. Information from referring agencies was described as variable and not especially useful when working with seriously troubled families, especially as it related to risk and child safety. Both groups indicated that the jargon of family preservation had permeated their agencies, and that working with other agencies was at times a challenge, though for different reasons. Finally, despite some reservations about the effectiveness of short-term treatment with families that face serious challenges, both groups of workers were generally satisfied with family preservation as an approach to practice.

Development and evaluation of a video-based modeling strategy for the dissemination phase of a school health tobacco -use prevention program

Even the best school health education programs will be unsuccessful if they are not disseminated effectively in a manner that encourages classroom adoption and implementation. This study involved two components: (1) the development of a videotape intervention to be used in the dissemination phase of a 4-year, NCI-funded diffusion study and (2) the evaluation of that videotape intervention strategy in comparison with a print (information transfer) strategy. Conceptualization has been guided by Social Learning Theory, Diffusion Theory, and communication theory. Additionally, the PRECEDE Framework has been used. Seventh and 8th grade classroom teachers from Spring Branch Independent School District in west Houston participated in the evaluation of the videotape and print interventions using a 57-item preadoption survey instrument developed by the UT Center for Health Promotion Research and Development. Two-way ANOVA was used to study individual score differences for five outcome variables: Total Scale Score (comprised of 57 predisposing, enabling, and reinforcing items), Adoption Characteristics Subscale, Attitude Toward Innovation Subscale, Receptivity Toward Innovation, and Reinforcement Subscale. The aim of the study is to compare the effect upon score differences of video and print interventions alone and in combination. Seventy-three 7th and 8th grade classroom teachers completed the study providing baseline and post-intervention measures on factors related to the adoption and implementation of tobacco-use prevention programs. Two-way ANOVA, in relation to the study questions, found significant scoring differences for those exposed to the videotape intervention alone for both the Attitude Toward Innovation Subscale and the Receptivity to Adopt Subscale. No significant results were found to suggest that print alone influences favorable scoring differences between baseline and post-intervention testing. One interaction effect was found suggesting video and print combined are more effective for influencing favorable scoring differences for the Reinforcement for the Adoption Subscale.^ This research is unique in that it represents a newly emerging field in health promotion communications research with implications for Social Learning Theory, Diffusion Theory, and communication science that are applicable to the development of improved school health interventions. ^

Decision evaluation in complex risk network systems (DecernsSDSS) demo and Black River case study

The selection of metrics for ecosystem restoration programs is critical for improving the quality of monitoring programs and characterizing project success. Moreover it is oftentimes very difficult to balance the importance of multiple ecological, social, and economical metrics. Metric selection process is a complex and must simultaneously take into account monitoring data, environmental models, socio-economic considerations, and stakeholder interests. We propose multicriteria decision analysis (MCDA) methods, broadly defined, for the selection of optimal sets of metrics to enhance evaluation of ecosystem restoration alternatives. Two MCDA methods, a multiattribute utility analysis (MAUT), and a probabilistic multicriteria acceptability analysis (ProMAA), are applied and compared for a hypothetical case study of a river restoration involving multiple stakeholders. Overall, the MCDA results in a systematic, unbiased, and transparent solution, informing restoration alternatives evaluation. The two methods provide comparable results in terms of selected metrics. However, because ProMAA can consider probability distributions for weights and utility values of metrics for each criteria, it is suggested as the best option if data uncertainty is high. Despite the increase in complexity in the metric selection process, MCDA improves upon the current ad-hoc decision practice based on the consultations with stakeholders and experts, and encourages transparent and quantitative aggregation of data and judgement, increasing the transparency of decision making in restoration projects. We believe that MCDA can enhance the overall sustainability of ecosystem by enhancing both ecological and societal needs.

Estrategias Avanzadas de Tabulación y Paralelismo en Programas Lógicos = Advanced Evaluation Strategies for Tabling and Parallelism in Logic Programs

Dentro de los paradigmas de programación en el mundo de la informática tenemos la "Programación Lógica'', cuyo principal exponente es el lenguaje Prolog. Los programas Prolog se componen de un conjunto de predicados, cada uno de ellos definido en base a reglas que aportan un elevado nivel de abstracción y declaratividad al programador. Sin embargo, las formulación con reglas implica, frecuentemente, que un predicado se recompute varias veces para la misma consulta y además, Prolog utiliza un orden fijo para evaluar reglas y objetivos (evaluación SLD) que puede entrar en "bucles infinitos'' cuando ejecuta reglas recursivas declarativamente correctas. Estas limitaciones son atacadas de raiz por la tabulación, que se basa en "recordar'' en una tabla las llamadas realizadas y sus soluciones. Así, en caso de repetir una llamada tendríamos ya disponibles sus soluciones y evitamos la recomputación. También evita "bucles infinitos'' ya que las llamadas que los generan son suspendidas, quedando a la espera de que se computen soluciones para las mismas. La implementación de la tabulación no es sencilla. En particular, necesita de tres operaciones que no pueden ser ejecutadas en tiempo constante simultáneamente. Dichas operaciones son: suspensión de llamadas, relanzamiento de llamadas y {acceso a variables. La primera parte de la tesis compara tres implementaciones de tabulación sobre Ciao, cada una de las cuales penaliza una de estas operaciones. Por tanto, cada solución tiene sus ventajas y sus inconvenientes y se comportan mejor o peor dependiendo del programa ejecutado. La segunda parte de la tesis mejora la funcionalidad de la tabulación para combinarla con restricciones y también para evitar computaciones innecesarias. La programación con restricciones permite la resolución de ecuaciones como medio de programar, mecanismo altamente declarativo. Hemos desarrollado un framework para combinar la tabulación con las restricciones, priorizando objetivos como la flexibilidad, la eficiencia y la generalidad de nuestra solución, obteniendo una sinergia entre ambas técnicas que puede ser aplicada en numerosas aplicaciones. Por otra parte, un aspecto fundamental de la tabulación hace referencia al momento en que se retornan las soluciones de una llamada tabulada. Local evaluation devuelve soluciones cuando todas las soluciones de la llamada tabulada han sido computadas. Por contra, batched evaluation devuelve las soluciones una a una conforme van siendo computadas, por lo que se adapta mejor a problemas donde no nos interesa encontrar todas las soluciones. Sin embargo, su consumo de memoria es exponencialmente peor que el de local evaluation. La tesis presenta swapping evaluation, que devuelve soluciones tan pronto como son computadas pero con un consumo de memoria similar a la de local evaluation. Además, se implementan operadores de poda, once/1, para descartar la búsqueda de soluciones alternativas cuando encontramos la solución deseada. Por último, Prolog adopta con relativa facilidad soluciones para paralelismo gracias a su flexibilidad en el control de la ejecución y a que sus asignaciones son lógicas. La tercera parte de la tesis extiende el paralelismo conjuntivo de Ciao para trabajar con programas no deterministas, lo que presenta dos problemas principales: los objetivos atrapados y la recomputación de objetivos. Las soluciones clásicas para los objetivos atrapados rompían muchos invariantes de la ejecución Prolog, siendo soluciones difíciles de mantener y de extender, que la experiencia nos dice que han caído en desuso. Nosotros proponemos una solución modular (basada en la implementación de swapping evaluation), localizada y que no rompe los invariantes de la ejecución Prolog, pero que mantiene un alto rendimiento de la ejecución paralela. En referencia a la recomputación de objetivos paralelos en presencia de no determinismo hemos adaptado ténicas derivadas de la tabulación para memorizar computaciones de estos objetivos y evitar su recomputación.

Social Housing in Great Metropolitan Areas: A New Methodology for Integrated Quality Evaluation and Comparison.

Este artículo expone la primera aplicación experimental de un método para el análisis y la comparación de experiencias de vivienda social internacional en el contexto de las grandes áreas metropolitanas. El método, desarrollado en la Universidad Politécnica Universidad Politécnica de Madrid (UPM) desde 2009, se basa en la utilización de códigos gráficos y numéricos comunes y la integración de diferentes escalas de aproximación al entorno construido, de la vivienda y su arquitectura, a sus materiales de construcción y llegando más lejos, la ciudad. Además, los datos están vinculados a tres conceptos clave estrechamente relacionados con las condiciones específicas de las grandes áreas metropolitanas: la economía, la densidad y la diversidad. El objetivo final de la investigación es proporcionar una herramienta para la evaluación de la calidad de las viviendas sociales, optimización de recursos y un diseño innovador.

Evaluation of three solar and daylighting control systems based on Calumen II, Ecotect and Radiance simulation programs to obtain an energy efficient and healthy interior in the experimental building prototype SDE10

Evaluation of three solar and daylighting control systems based on Calumen II, Ecotect and Radiance simulation programs to obtain an energy efficient and healthy interior in the experimental building prototype SDE10

The influence of transparency on self-evaluation as part of the university accreditation process in Peru.

In order to show the choice of transparency as the guiding principle of the accreditation process, the article evaluates its influence on the fundamental subprocess of self-evaluation, thereby confirming that transparency is an essential tool for continuous improvement of academic processes and those of educational quality management. It fosters educational innovation and permits the sustainability of the continuous accreditation process over time, resulting in greater probabilities of university self-regulation through systemization of the process, with the objective of continuous improvement of university degree programs. The article analyzes the influence of transparency on each activity of the self-evaluation process according to the Peruvian accreditation model prepared under the total quality approach, as a reference for other accreditation models, proposing concrete transparency actions and evaluating its influence on the stakeholder groups in the self-evaluation process, as well as on the efficiency and effectiveness of the process. It is concluded that transparency has a positive influence on the training of human capital and the formation of the university?s organizational culture, facilitating dissemination, understanding and involvement of the stakeholder groups in the continuous improvement of accreditation activities and increasing their acceptance of change and commitment to the process. It is confirmed that transparency contributes toward increasing the efficiency index of the self-evaluation process by reducing operating costs through adequate, accessible, timely contribution of information by the stakeholders and through the optimization of the time spent gathering relevant information. In addition, it is concluded that transparency contributes toward increasing the effectiveness index of self-evaluation by facilitating the achievement of its objectives through synthetic, useful, reliable interpretation of the education situation and the formulation of feasible improvement plans based on the adequacy, relevance, visibility, pertinence and truthfulness of the information analyzed.