A comprehensive framework for evaluating funder's performance

Evaluation methods for assessing the performance of non-profit funders are lacking. The purpose of the research was to create a comprehensive framework that systematically assesses the goals and objectives of a funder, how these relate to the funder's allocation of resources, and the potential impact of programs and services selected by the funder for resource allocation to address organizational goals and objectives. The Houston Affiliate of Susan G. Komen for the Cure, a local chapter of a national breast cancer awareness advocacy organization, was selected as the funding agency whose performance assessment was to assist in the creation of this framework. Evaluation approaches from the government sector were adapted and incorporated into the research to guide the methods used to answer the three research questions corresponding to the three phases of research within the study: (1) what are the funding goals and objectives of the Affiliate?; (2) what allocation scheme does the organization use to address these goals and objectives and select programs for funding?; and, (3) to what extent do the programs funded by the Affiliate have potential long-term impact? ^ Within the first stage of the research, document reviews of the Affiliate's mission-based documents and bylaws and interviews with organizational and community informants revealed a highly latent constellation of broad objectives that were not formalized into one guiding document, thus creating gaps in management and governance. Within the second phase of the research, reviews of grant applications from the 2008-2009 funding cycle and interviews with employees and volunteers familiar with the funding process revealed competing ideas regarding resource allocation in light of vague organizational documents describing funding goals and objectives. Within the final stage of the research, these findings translated to the Affiliate selecting programs with highly varying potential long-term impact with regards to addressing goals and objectives relating to breast cancer education, screening, diagnostics, treatment, and support. The resulting performance assessment framework, consisting of three phases of research utilizing organizational documents and key informant interviews, demonstrated the importance of clearly defined funding goals and objectives, reference documents and committee participation within the funding process, and regular reviews of potential long-term impact for selected programs, all supported by the active participation and governance of a funder's Board of Directors.^

An evaluation of sleep hypnotic and stimulant effect on human performance using a computerized psychological test battery

This study evaluated the administration-time-dependent effects of a stimulant (Dexedrine 5-mg), a sleep-inducer (Halcion 0.25-mg) and placebo (control) on human performance. The investigation was conducted on 12 diurnally active (0700-2300) male adults (23-38 yrs) using a double-blind, randomized sixway-crossover three-treatment, two-timepoint (0830 vs 2030) design. Performance tests were conducted hourly during sleepless 13-hour studies using a computer generated, controlled and scored multi-task cognitive performance assessment battery (PAB) developed at the Walter Reed Army Institute of Research. Specific tests were Simple and Choice Reaction Time, Serial Addition/Subtraction, Spatial Orientation, Logical Reasoning, Time Estimation, Response Timing and the Stanford Sleepiness Scale. The major index of performance was "Throughput", a combined measure of speed and accuracy.^ For the Placebo condition, Single and Group Cosinor Analysis documented circadian rhythms in cognitive performance for the majority of tests, both for individuals and for the group. Performance was best around 1830-2030 and most variable around 0530-0700 when sleepiness was greatest (0300).^ Morning Dexedrine dosing marginally enhanced performance an average of 3% with reference to the corresponding in time control level. Dexedrine AM also increased alertness by 10% over the AM control. Dexedrine PM failed to improve performance with reference to the corresponding PM control baseline. With regard to AM and PM Dexedrine administrations, AM performance was 6% better with subjects 25% more alert.^ Morning Halcion administration caused a 7% performance decrement and 16% increase in sleepiness and a 13% decrement and 10% increase in sleepiness when administered in the evening compared to corresponding in time control data. Performance was 9% worse and sleepiness 24% greater after evening versus morning Halcion administration.^ These results suggest that for evening Halcion dosing, the overnight sleep deprivation occurring in coincidence with the nadir in performance due to circadian rhythmicity together with the CNS depressant effects combine to produce performance degradation. For Dexedrine, morning administration resulted in only marginal performance enhancement; Dexedrine in the evening was less effective, suggesting the 5-mg dose level may be too low to counteract the partial sleep deprivation and nocturnal nadir in performance. ^

Territorial quality and sustainable development : Indicators of performance in the context of a model for participatory rural tourism certification

El turismo rural ha sido incorporado por pequeños establecimientos agropecuarios del partido de Cnel. Suárez, provincia de Buenos Aires, Argentina, como actividad alternativa para mejorar la calidad de vida de sus miembros y superar situaciones de crisis agravadas por la marginalidad productiva del SO bonaerense, región a la que pertenecen. Bajo el programa Cambio Rural del INTA (Instituto Nacional de Tecnología Agropecuaria) conforman el Grupo 'Cortaderas II', junto a otros emprendedores interesados en valorar el medio rural. Han avanzado en el proceso de reconocimiento de su identidad y puesta en valor de recursos específicos con anclaje en el territorio. Esta identidad comienza a apreciarse internamente, a raíz de la dinámica grupal lograda y la incipiente articulación con otros actores para la construcción de un partenariado público y privado que genere sinergias y contribuya al desarrollo sustentable del territorio. Sin embargo, aún no es claramente percibida por el turista, cada vez más exigente. Por lo tanto, el presente trabajo persigue proponer indicadores para evaluar el desempeño de un Sistema de Gestión de Calidad con enfoque territorial que, adaptando el modelo europeo 'Marca de Calidad Territorial', sustente una estrategia comercial de diferenciación del servicio y simultáneamente, mida el progreso hacia una mejor calidad de vida y fortalecimiento de vínculos con la cultura local y el entorno físico-natural en el marco del desarrollo sustentable. La investigación se plantea para la micro escala, ya que se trata de un estudio de caso, relevándose información primaria mediante observación directa y entrevistas semi-estructuradas, complementada con información secundaria diagnóstica utilizada por INTA. Las características del grupo y su dinámica de funcionamiento bajo el programa Cambio Rural revelan que es posible adoptar un proceso de certificación participativa propuesto para cuatro pilares de la calidad: de Bienes y Servicios, Institucional, Social y Ambiental. El modelo se integra con indicadores de evaluación de desempeño, agrupados en áreas clave para cada una de las dimensiones de la sustentabilidad, que contemplan el paisaje y la gestión de los recursos naturales; el impacto económico de la actividad, la calidad de la oferta y satisfacción del turista; así como las relaciones sociales internas y los vínculos con otros actores del territorio. Principalmente se encontraron fortalezas en la búsqueda de partenariados y debilidades en aspectos de comunicación y promoción. Se considera que este sistema de herramientas de gestión sustentable permitiría superar las dificultades de una certificación individual, pudiendo aplicarse a emprendimientos con otra ubicación geográfica

Territorial quality and sustainable development : Indicators of performance in the context of a model for participatory rural tourism certification

El turismo rural ha sido incorporado por pequeños establecimientos agropecuarios del partido de Cnel. Suárez, provincia de Buenos Aires, Argentina, como actividad alternativa para mejorar la calidad de vida de sus miembros y superar situaciones de crisis agravadas por la marginalidad productiva del SO bonaerense, región a la que pertenecen. Bajo el programa Cambio Rural del INTA (Instituto Nacional de Tecnología Agropecuaria) conforman el Grupo 'Cortaderas II', junto a otros emprendedores interesados en valorar el medio rural. Han avanzado en el proceso de reconocimiento de su identidad y puesta en valor de recursos específicos con anclaje en el territorio. Esta identidad comienza a apreciarse internamente, a raíz de la dinámica grupal lograda y la incipiente articulación con otros actores para la construcción de un partenariado público y privado que genere sinergias y contribuya al desarrollo sustentable del territorio. Sin embargo, aún no es claramente percibida por el turista, cada vez más exigente. Por lo tanto, el presente trabajo persigue proponer indicadores para evaluar el desempeño de un Sistema de Gestión de Calidad con enfoque territorial que, adaptando el modelo europeo 'Marca de Calidad Territorial', sustente una estrategia comercial de diferenciación del servicio y simultáneamente, mida el progreso hacia una mejor calidad de vida y fortalecimiento de vínculos con la cultura local y el entorno físico-natural en el marco del desarrollo sustentable. La investigación se plantea para la micro escala, ya que se trata de un estudio de caso, relevándose información primaria mediante observación directa y entrevistas semi-estructuradas, complementada con información secundaria diagnóstica utilizada por INTA. Las características del grupo y su dinámica de funcionamiento bajo el programa Cambio Rural revelan que es posible adoptar un proceso de certificación participativa propuesto para cuatro pilares de la calidad: de Bienes y Servicios, Institucional, Social y Ambiental. El modelo se integra con indicadores de evaluación de desempeño, agrupados en áreas clave para cada una de las dimensiones de la sustentabilidad, que contemplan el paisaje y la gestión de los recursos naturales; el impacto económico de la actividad, la calidad de la oferta y satisfacción del turista; así como las relaciones sociales internas y los vínculos con otros actores del territorio. Principalmente se encontraron fortalezas en la búsqueda de partenariados y debilidades en aspectos de comunicación y promoción. Se considera que este sistema de herramientas de gestión sustentable permitiría superar las dificultades de una certificación individual, pudiendo aplicarse a emprendimientos con otra ubicación geográfica

Territorial quality and sustainable development : Indicators of performance in the context of a model for participatory rural tourism certification

El turismo rural ha sido incorporado por pequeños establecimientos agropecuarios del partido de Cnel. Suárez, provincia de Buenos Aires, Argentina, como actividad alternativa para mejorar la calidad de vida de sus miembros y superar situaciones de crisis agravadas por la marginalidad productiva del SO bonaerense, región a la que pertenecen. Bajo el programa Cambio Rural del INTA (Instituto Nacional de Tecnología Agropecuaria) conforman el Grupo 'Cortaderas II', junto a otros emprendedores interesados en valorar el medio rural. Han avanzado en el proceso de reconocimiento de su identidad y puesta en valor de recursos específicos con anclaje en el territorio. Esta identidad comienza a apreciarse internamente, a raíz de la dinámica grupal lograda y la incipiente articulación con otros actores para la construcción de un partenariado público y privado que genere sinergias y contribuya al desarrollo sustentable del territorio. Sin embargo, aún no es claramente percibida por el turista, cada vez más exigente. Por lo tanto, el presente trabajo persigue proponer indicadores para evaluar el desempeño de un Sistema de Gestión de Calidad con enfoque territorial que, adaptando el modelo europeo 'Marca de Calidad Territorial', sustente una estrategia comercial de diferenciación del servicio y simultáneamente, mida el progreso hacia una mejor calidad de vida y fortalecimiento de vínculos con la cultura local y el entorno físico-natural en el marco del desarrollo sustentable. La investigación se plantea para la micro escala, ya que se trata de un estudio de caso, relevándose información primaria mediante observación directa y entrevistas semi-estructuradas, complementada con información secundaria diagnóstica utilizada por INTA. Las características del grupo y su dinámica de funcionamiento bajo el programa Cambio Rural revelan que es posible adoptar un proceso de certificación participativa propuesto para cuatro pilares de la calidad: de Bienes y Servicios, Institucional, Social y Ambiental. El modelo se integra con indicadores de evaluación de desempeño, agrupados en áreas clave para cada una de las dimensiones de la sustentabilidad, que contemplan el paisaje y la gestión de los recursos naturales; el impacto económico de la actividad, la calidad de la oferta y satisfacción del turista; así como las relaciones sociales internas y los vínculos con otros actores del territorio. Principalmente se encontraron fortalezas en la búsqueda de partenariados y debilidades en aspectos de comunicación y promoción. Se considera que este sistema de herramientas de gestión sustentable permitiría superar las dificultades de una certificación individual, pudiendo aplicarse a emprendimientos con otra ubicación geográfica

Specific skills assessment learned by students of building engineering and technical architecture in topics relating to durability and control of concrete structures

In many university courses such as Building Engineering or Technical Architectural, the high density of the contents included in the curriculum, make the student, after graduation, unable to develop the skills already acquired and evaluated in the disciplines of the first courses. From the Group of Educational Innovation at the Polytechnic University of Madrid (UPM) "Teaching of Structural Concrete" (GIEHE) we have conducted a study in which are valued specific skills acquired by students after the first courses of career. We have worked with students from UPM fourth-year career and with Technical Architecture students who have completed their studies and also have completed the Adaptation Course of Technical Architecture to the Building Engineer. The work is part of the Educational Innovation Project funded by the UPM "Integration of training and assessment of generic and specific skills in structural concrete" We have evaluated specific skills learned in the areas of durability and control of structural concrete structures. The results show that overall, students are not able to fully develop the skills already acquired earlier, even being these essential to their professional development. Possibly, the large amount of content taught in these degrees together with a teaching and assessment of "flat profile", ie, which are presented and evaluated with the same intensity as the fundamental and the accessory, are causes enough to cause these results.

La evaluación del aprendizaje de los deportes colectivos en educación física a través del modelo de enseñanza comprensivo

El presente trabajo tiene como objetivo evaluar el aprendizaje de los deportes colectivos en el alumnado de segundo ciclo de Educación Secundaria Obligatoria (ESO) utilizando una herramienta denominada Game Performance Assessment Instrument (GPAI). Para ello se ha realizado una revisión de dos modelos de enseñanza existentes: el modelo técnico y el modelo comprensivo. Así mismo hemos revisado dos tipos de procesos de evaluación; por un lado la evaluación sumativa, más relacionada con el modelo técnico; por otro lado la evaluación formativa, más próxima al modelo comprensivo. A continuación se ha llevado a cabo una intervención didáctica aplicando el modelo de enseñanza comprensivo (TGfU) y el instrumento mencionado anteriormente (GPAI). Los resultados obtenidos han sido utilizados para proponer un sistema de evaluación basado en el modelo comprensivo con la utilización del GPAI y a través de la coevaluación como forma de asentar el aprendizaje en el alumnado. Se concluye que el modelo de enseñanza comprensivo (TGfU) con el GPAI y una evaluación entre iguales es bastante óptimo para favorecer el proceso de enseñanzaaprendizaje de los deportes colectivos en el alumnado de segundo ciclo de Educación Secundaria Obligatoria (ESO). ABSTRACT The present work has the aim to assess the learning of collective sports in Secondary Education students using a tool called Game Performance Assessment Instrument (GPAI). For that we have used to revision of two teaching models: the technical model and Teaching Games for Understanding (TGfU). At the same time, we have reviewed two kinds of assessment process; on one hand summative assessment which is related to technical model; on the other hand formative assessment nearest to Teaching Games for Understanding (TGfU). Next it has been done a didactic intervention using the Teaching Games for Understanding (TGfU) and that instrument mentioned beforehand. The results obtained have been used to suggest an assessment system based on Teaching Games for Understanding (TGfU) using the GPAI through the peer assessment as a way of teaching.

A framework for assessing business-incubation input, process and output performance:proceedings of the RENT XXVII Conference, 20-22 November 2013, Vilnius, Lithuania

With business incubators deemed as a potent infrastructural element for entrepreneurship development, business incubation management practice and performance have received widespread attention. However, despite this surge of interest, scholars have questioned the extent to which business incubation delivers added value. Thus, there is a growing awareness among researchers, practitioners and policy makers of the need for more rigorous evaluation of the business incubation output performance. Aligned to this is an increasing demand for benchmarking business incubation input/process performance and highlighting best practice. This paper offers a business incubation assessment framework, which considers input/process and output performance domains with relevant indicators. This tool adds value on different levels. It has been developed in collaboration with practitioners and industry experts and therefore it would be relevant and useful to business incubation managers. Once a large enough database of completed questionnaires has been populated on an online platform managed by a coordinating mechanism, such as a business incubation membership association, business incubator managers can reflect on their practices by using this assessment framework to learn their relative position vis-à-vis their peers against each domain. This will enable them to align with best practice in this field. Beyond implications for business incubation management practice, this performance assessment framework would also be useful to researchers and policy makers concerned with business incubation management practice and impact. Future large-scale research could test for construct validity and reliability. Also, discriminant analysis could help link input and process indicators with output measures.

Hierarchical control and performance regimes in local government

Purpose – The purpose of this paper is to consider hierarchical control as a mode of governance, and analyses the extent of control exhibited by central government over local government through the best value (BV) and comprehensive performance assessment (CPA) performance regimes. Design/methodology/approach – This paper utilises Ouchi's framework and, specifically, his articulation of bureaucratic or hierarchical control in the move towards achievement of organisational objectives. Hierarchical control may be inferred from the extent of “command and control” by Central Government, use of rewards and sanctions, and alignment to government priorities and discrimination of performance. Findings – CPA represents a more sophisticated performance regime than BV in the governance of local authorities by central government. In comparison to BV, CPA involved less scope for dialogue with local government prior to introduction, closer inspection of and direction of support toward poorer performing authorities, and more alignment to government priorities in the weightings attached to service blocks. Originality/value - The paper focuses upon the hierarchic/bureaucratic mode of governance as articulated by Ouchi and expands on this mode in order to analyse shifts in performance regimes in the public sector.

Teaching Strategies and Ontologies for E-Learning

* The work is partially suported by Russian Foundation for Basic Studies (grant 02-01-00466).

Performance measurement with multiple interrelated variables and threshold target levels:evidence from retail firms in the US

In this study, we developed a DEA-based performance measurement methodology that is consistent with performance assessment frameworks such as the Balanced Scorecard. The methodology developed in this paper takes into account the direct or inverse relationships that may exist among the dimensions of performance to construct appropriate production frontiers. The production frontiers we obtained are deemed appropriate as they consist solely of firms with desirable levels for all dimensions of performance. These levels should be at least equal to the critical values set by decision makers. The properties and advantages of our methodology against competing methodologies are presented through an application to a real-world case study from retail firms operating in the US. A comparative analysis between the new methodology and existing methodologies explains the failure of the existing approaches to define appropriate production frontiers when directly or inversely related dimensions of performance are present and to express the interrelationships between the dimensions of performance.

Az egyéni teljesítményértékelési rendszer működésének tapasztalatai a magyar közigazgatásban (Operating experiences of the individual performance appraisal system in the Hungarian public sector)

A tanulmány – egy 2005 végén lefolytatott, országosan reprezentatív kérdőíves felmérés empirikus bázisán – a magyar közigazgatási dolgozók körében 2002-től bevezetett egyéni teljesítményértékelési rendszert vizsgálja. A rendszer egészét tekintve a vizsgálat végkövetkeztetései nem túl rózsásak: az eredmények arra utalnak, hogy a rendszer mint egész nem képes kitűzött fő célját, a munkateljesítmények javítását elérni. Az egyes részrendszerek, szervezettípusok szintjén differenciáltabb eredményeket kapunk, míg a minisztériumokban a rendszer egésze a szándékolttól alapvetően különböző célokat látszik szolgálni, addig a többi szervezettípusnál inkább „csak” a rendszer tökéletlen működéséről beszélhetünk. ____ On the basis of a questionnaire survey of a nationally representative sample of Hungarian civil servants, the study attempts to give a preliminary evaluation of the individual performance assessment system introduced in 2002 throughout the Hungarian civil service. The general conclusion of the analysis is that the performance assessment system is unlikely to reach its goals: its central element, the performance related pay incentives are overly dispersed and, on the average, insignificant in size, meanwhile the objectivity of performance assessments is also questionable. Moreover, comparative analysis of responses from different administrative branches reveals an interesting idiosyncrasy characteristic for central government ministries. Here, the main function of the incentive system seems to be ensuring the labor market competitiveness of the ministries as employers by enabling them to pay higher-than-usual salaries for employees having certain types of expertise.

The effects of the outdoor experiential ropes course on perceived team performance

The purpose of this quasi-experimental study was to explore the perceived effects of the Ropes Course on the performance of intact work teams. The dependent variable, team performance, was measured by the Team Performance Assessment, a 20 question inventory. The Ropes Course, the independent variable, was an outdoor experiential training program presently marketed as a highly effective team building training program. Issues the team addressed in the highly emotional and physical environment were purported to transfer back to the work environment and act as a catalyst for change The Ropes Course in this study consisted of a day long series of outdoor mental, emotional and physical exercises addressing the issues of goal-setting, role expectations, accountability, trust, respect, communication, problem-solving and decision-making. The 68 subjects, 37 in the treatment group and 31 in the control were employees of a large international financial institution. They were not chosen by random selection. The work teams' managers recognized a need to improve team morale, performance and functioning due to corporate reengineering and downsizing resulting in team members' job losses. Control teams were partially matched to the treatment teams on the basis of professional composition and similar job descriptions. The pretest of the Team Performance Assessment was given the morning of the Ropes Course treatment and the posttest was given three to five weeks later. The control teams received the pretests and posttests at about the same time intervals at their work location but received no Ropes Course treatment. The treatment teams' scores and the control teams' scores were statistically compared using the Multivariate Analysis of Variance (MANOVA) and the Multivariate Analysis of Covariance (MANCOVA) at the .05 level of significance. The statistical analysis revealed a significant difference between the control and experimental teams after the team building Ropes Course training as measured by the Team Performance Assessment (Gilbert, 1996). ^

Promoção, vencimento e avaliação de desempenho do professor do ensino fundamental: o PCCR da rede municipal de ensino de Natal/RN (2004-2010)

This study is about the enhancement of the elementary school in Natal/RN, (PCCR - Law No. 058/2004), concerning to the horizontal promotion through the performance evaluation. It uses as reference the education policy on the legislative field hegemony and the managerial model. The analysis of the teacher´s valorization is based on the policy of Funds (Fundef and Fundeb) in the Brazilian social and educational agenda. The study focuses on the Career Plan, Career and Remuneration of teachers in the period (2004-2010).The study focuses on the Career Plan, Career and Remuneration of teachers in the period (2004-2010). The thesis argues for the necessity to adopt a direct relationship between career development and horizontal promotion for more others fifteen classes, no matter to any conditioning variables. In addition, the performance shall be evaluated by interval of 25 years to reach at least the provisions decided in the law, which determines the salary adjustment in 5% at every two years, as pointed at the PCCR, about teachers remuneration, and teachers qualifications. A work of a bibliographic and a documental review about the education funding with the purpose of enhancement of educational work, career concepts, and also promotion and evaluation performance as well was performed based on experts authors in this field. The survey was organized with the aim of articulating quantitative and qualitative information, analyzing data from the teacher's salary - payrolls and paychecks - also applying a questionnaire. After the implementation of the PCCR, it was found that the wage indices for horizontal promotion during the teaching career are tied to a strategy for evaluating the performance which disqualifies the teacher‟s salaries in a minimum percentage of 25% (up to 25 years) and there are also elements that disturb the promotion strategy. The national minimum wage was set in three salaries by the PSPN Lei nº11.738/2008 but it never reaches the three salaries at Natal/RN educational system.Otherwise, the elements that structure the horizontal promotion in fifteen classes, throughout the career, flout the minimum years of teaching work, long established in 25 years. In addition, changes in terms in the salary increase depend on individual efforts by professional development through titration. Concerning to the career, despite of the category approving its PCCR, neither this instrument nor the Funds Policy managed to establish regulations were able to cope effective rules for valuing the teachers in the educational district system. It is necessary to ensure, in percentage terms and financial, the real remuneration of teachers with the attainment of horizontal promotion, reviewing the elements that structure the career and the determinants of performance evaluation.

A pilot study of orthopaedic resident self-assessment using a milestones' survey just prior to milestones implementation.

OBJECTIVE: To pilot test if Orthopaedic Surgery residents could self-assess their performance using newly created milestones, as defined by the Accreditation Council on Graduate Medical Education. METHODS: In June 2012, an email was sent to Program Directors and administrative coordinators of the 154 accredited Orthopaedic Surgery Programs, asking them to send their residents a link to an online survey. The survey was adapted from the Orthopaedic Surgery Milestone Project. Completed surveys were aggregated in an anonymous, confidential database. SAS 9.3 was used to perform the analyses. RESULTS: Responses from 71 residents were analyzed. First and second year residents indicated through self-assessment that they had substantially achieved Level 1 and Level 2 milestones. Third year residents reported they had substantially achieved 30/41, and fourth year residents, all Level 3 milestones. Fifth year, graduating residents, reported they had substantially achieved 17 Level 4 milestones, and were extremely close on another 15. No milestone was rated at Level 5, the maximum possible. Earlier in training, Patient Care and Medical Knowledge milestones were rated lower than the milestones reflecting the other four competencies of Practice Based Learning and Improvement, Systems Based Practice, Professionalism, and Interpersonal Communication. The gap was closed by the fourth year. CONCLUSIONS: Residents were able to successfully self-assess using the 41 Orthopaedic Surgery milestones. Respondents' rate improved proficiency over time. Graduating residents report they have substantially, or close to substantially, achieved all Level 4 milestones. Milestone self-assessment may be a useful tool as one component of a program's overall performance assessment strategy.