Program evaluation primer: A review of three evaluations

Program evaluation—a “tool used to assess the implementation and outcomes of a program, to increase a program’s efficiency and impact over time, and to demonstrate accountability” (MacDonald et. al, 2001, p. 1)—is an essential process to program assessment and improvement. This paper overviews three published program evaluations and considers important aspects of program evaluation more broadly.

Who is in charge? An L2 Discourse Intonation Study on Four Prosodic Parameters to Exert the Pragmatic Function of Dominance and Control in the Context of L2 Non-specialist Public Speaking

This paper reports the findings from a study of the learning of English intonation by Spanish speakers within the discourse mode of L2 oral presentation. The purpose of this experiment is, firstly, to compare four prosodic parameters before and after an L2 discourse intonation training programme and, secondly, to confirm whether subjects, after the aforementioned L2 discourse intonation training, are able to match the form of these four prosodic parameters to the discourse-pragmatic function of dominance and control. The study designed the instructions and tasks to create the oral and written corpora and Brazil’s Pronunciation for Advanced Learners of English was adapted for the pedagogical aims of the present study. The learners’ pre- and post-tasks were acoustically analysed and a pre / post- questionnaire design was applied to interpret the acoustic analysis. Results indicate most of the subjects acquired a wider choice of the four prosodic parameters partly due to the prosodically-annotated transcripts that were developed throughout the L2 discourse intonation course. Conversely, qualitative and quantitative data reveal most subjects failed to match the forms to their appropriate pragmatic functions to express dominance and control in an L2 oral presentation.

Fast and robust face finding via local context

[EN]In visual surveillance face detection can be an important cue for initializing tracking algorithms. Recent work in psychophics hints at the importance of the local context of a face for robust detection, such as head contours and torso. This paper describes a detector that actively utilizes the idea of local context. The promise is to gain robustness that goes beyond the capabilities of traditional face detection making it particularly interesting for surveillance. The performance of the proposed detector in terms of accuracy and speed is evaluated on data sets from PETS 2000 and PETS 2003 and compared to the object-centered approach. Particular attention is paid to the role of available image resolution.

Sanctity of Human Life in the Context of Human Genetic Research

The study on the concept of sanctity of human life is a journey in finding out what is it said to be “human” in human life. It is an evaluation of the universal concept and the role it plays in controlling and moulding human conduct and relationships. This concept is a foundational principle of human rights law and the grundnorm of every legal system. However, of late, the challenges by way of certain advances in human genetic research had prompted the need to evaluate the significance and extent of the concept in human endeavours. Scientific advances by way of human genetic research promises significant diagnostic and therapeutic advances but at the same time pose threat to fundamental notions and assumptions on humanity, hence there is a global concern to derive common legal standards, Thus the major challenge is to analyse universal principles which can be a common criteria for evolving legal standards to control certain advances in human genetic research. Hence the relevance of the study. The study aims at analysing the content, scope, extent and limitation of the concept of sanctity of human life. In this attempt it evaluates the extent to which the concept had been accommodated by legal systems and international human rights regimes. The problem which had been undertaken in the study is the extent of intrusion made to the concept by virtue of certain advances in human genetic research.

Evaluation use within a humanitarian non-governmental organization’s health care user-fee exemption program in West Africa

L’évaluation de l’action humanitaire (ÉAH) est un outil valorisé pour soutenir l’imputabilité, la transparence et l’efficience de programmes humanitaires contribuant à diminuer les inéquités et à promouvoir la santé mondiale. L’EAH est incontournable pour les parties prenantes de programme, les bailleurs de fonds, décideurs et intervenants souhaitant intégrer les données probantes aux pratiques et à la prise de décisions. Cependant, l’utilisation de l’évaluation (UÉ) reste incertaine, l’ÉAH étant fréquemment menée, mais inutilisé. Aussi, les conditions influençant l’UÉ varient selon les contextes et leur présence et applicabilité au sein d’organisations non-gouvernementales (ONG) humanitaires restent peu documentées. Les évaluateurs, parties prenantes et décideurs en contexte humanitaire souhaitant assurer l’UÉ pérenne détiennent peu de repères puisque rares sont les études examinant l’UÉ et ses conditions à long terme. La présente thèse tend à clarifier ces enjeux en documentant sur une période de deux ans l’UÉ et les conditions qui la détermine, au sein d’une stratégie d’évaluation intégrée au programme d’exemption de paiement des soins de santé d’une ONG humanitaire. L’objectif de ce programme est de faciliter l’accès à la santé aux mères, aux enfants de moins de cinq ans et aux indigents de districts sanitaires au Niger et au Burkina Faso, régions du Sahel où des crises alimentaires et économiques ont engendré des taux élevés de malnutrition, de morbidité et de mortalité. Une première évaluation du programme d’exemption au Niger a mené au développement de la stratégie d’évaluation intégrée à ce même programme au Burkina Faso. La thèse se compose de trois articles. Le premier présente une étude d’évaluabilité, étape préliminaire à la thèse et permettant de juger de sa faisabilité. Les résultats démontrent une logique cohérente et plausible de la stratégie d’évaluation, l’accessibilité de données et l’utilité d’étudier l’UÉ par l’ONG. Le second article documente l’UÉ des parties prenantes de la stratégie et comment celle-ci servit le programme d’exemption. L’utilisation des résultats fut instrumentale, conceptuelle et persuasive, alors que l’utilisation des processus ne fut qu’instrumentale et conceptuelle. Le troisième article documente les conditions qui, selon les parties prenantes, ont progressivement influencé l’UÉ. L’attitude des utilisateurs, les relations et communications interpersonnelles et l’habileté des évaluateurs à mener et à partager les connaissances adaptées aux besoins des utilisateurs furent les conditions clés liées à l’UÉ. La thèse contribue à l’avancement des connaissances sur l’UÉ en milieu humanitaire et apporte des recommandations aux parties prenantes de l’ONG.

Early Bird & Night Owl Evaluation Tool (EBNOET) v2015

The early bird and night owl restaurant tool found in the accompanying Excel file provides an estimate of the effects of offering off-peak special menu prices. Unlike the classic back-of-envelope calculation, the tool includes the effect of anticipated cannibalization of full-price covers and seeks to optimize table use. The tool also considers the revenue from new customers attracted by the early bird or night owl promotions, as well as the level of increased business needed to achieve the net monetary value target for the promotion.

Manipulation in U.S. REIT Investment Performance Evaluation: Empirical Evidence

We investigate whether Real Estate Investment Trust (REIT) managers actively manipulate performance measures in spite of the strict regulation under the REIT regime. We provide empirical evidence that is consistent with this hypothesis. Specifically, manipulation strategies may rely on the opportunistic use of leverage. However, manipulation does not appear to be uniform across REIT sectors and seems to become more common as the level of competition in the underlying property sector increases. We employ a set of commonly used traditional performance measures and a recently developed manipulation-proof measure (MPPM, Goetzmann, Ingersoll, Spiegel, and Welch (2007)) to evaluate the performance of 147 REITs from seven different property sectors over the period 1991-2009. Our findings suggest that the existing REIT regulation may fail to mitigate a substantial agency conflict and that investors can benefit from evaluating return information carefully in order to avoid potentially manipulative funds.

Evaluation of a Quality Improvement (QI) Intervention for Maternal, Newborn, and Child Health (MNCH) in Colombia: The Mil Días Intervention of SINERGIAS in the Department of Bolívar

Thesis (Master's)--University of Washington, 2016-06

Web-based museum trails on PDAs for university-level design students: Design and evaluation

This paper describes the development and evaluation of web-based museum trails for university-level design students to access on handheld devices in the Victoria and Albert Museum (V&A) in London. The trails offered students a range of ways of exploring the museum environment and collections, some encouraging students to interpret objects and museum spaces in lateral and imaginative ways, others more straightforwardly providing context and extra information. In a three-stage qualitative evaluation programme, student feedback showed that overall the trails enhanced students’ knowledge of, interest in, and closeness to the objects. However, the trails were only partially successful from a technological standpoint due to device and network problems. Broader findings suggest that technology has a key role to play in helping to maintain the museum as a learning space which complements that of universities as well as schools. This research informed my other work in visitor-constructed learning trails in museums, specifically in the theoretical approach to data analysis used, in the research design, and in informing ways to structure visitor experiences in museums. It resulted in a conference presentation, and more broadly informed my subsequent teaching practice.

Material Evaluation of an Elastomer, Epoxy and Lightweight Concrete Rail Attachment System for Direct Fixation Light Rail Applications

Thesis (Master's)--University of Washington, 2016-08

New Framework and Metrics for Economic Evaluation of Medical Technology in the Presence of Heterogeneity and Its Application to the Evaluation of Bariatric Surgery

Thesis (Ph.D.)--University of Washington, 2016-07

Prosocial Behaviour in Palliative Nurses: Psychometric Evaluation of the Prosociality Scale

Aim: The aim of this study was to examine the psychometric properties of a prosociality scale within the palliative nursing context, and then examine the impact of prosocial behaviour in relation to job and educational satisfaction among palliative nurses. Methods: An online cross-sectional survey was conducted in 25 Italian palliative care centres, with a total of 107 nurses completing the prosociality scale by Caprara et al (2005). Exploratory and confirmatory factor analyses were examined to evaluate a multidimensional model of prosociality. Results: A three-factor solution with a second order factor fitted the data well. The three dimensions extracted were labelled as helping, empathy, and sharing. Participants reported high levels of prosociality. In addition, prosociality was positively associated with job and educational satisfaction. Conclusions: The prosociality scale was valid and reliable when tested with palliative nurses. Although prosociality may be embedded in nurses’ personalities, this quality should be actively promoted to expand and improve the culture and the ethics of nursing.

Context-aware framework for collaborative applications

Future pervasive environments will take into consideration not only individual user’s interest, but also social relationships. In this way, pervasive communities can lead the user to participate beyond traditional pervasive spaces, enabling the cooperation among groups and taking into account not only individual interests, but also the collective and social context. Social applications in CSCW (Computer Supported Cooperative Work) field represent new challenges and possibilities in terms of use of social context information for adaptability in pervasive environments. In particular, the research describes the approach in the design and development of a context.aware framework for collaborative applications (CAFCA), utilizing user’s context social information for proactive adaptations in pervasive environments. In order to validate the proposed framework an evaluation was conducted with a group of users based on enterprise scenario. The analysis enabled to verify the impact of the framework in terms of functionality and efficiency in real-world conditions. The main contribution of this thesis was to provide a context-aware framework to support collaborative applications in pervasive environments. The research focused on providing an innovative socio-technical approach to exploit collaboration in pervasive communities. Finally, the main results reside in social matching capabilities for session formation, communication and coordinations of groupware for collaborative activities.

Evaluation of online higher education: learning, interaction and technology

The work presents a theoretical framework for the evaluation of e-Teaching that aims at positioning the online activities designed and developed by the teacher as to the Learning, Interaction and Technology Dimensions. The theoretical research that underlies the study was developed reflecting current thinking on the promotion of quality of teaching and of the integration of information and communication tools into the curriculum in Higher Education (HE), i.e., bearing in mind some European guidelines and policies on this subject. This way, an answer was sought to be given to one of the aims put forward in this study, namely to contribute towards the development of a conceptual framework to support research on evaluation of e-teaching in the context of HE. Based on the theoretical research carried out, an evaluation tool (SCAI) was designed, which integrates the two questionnaires developed to collect the teachers' and the students' perceptions regarding the development of e-activities. Consequently, an empirical study was structured and carried out, allowing SCAI tool to be tested and validated in real cases. From the comparison of the theoretical framework established and the analysis of the data obtained, we found that the differences in teaching should be valued and seen as assets by HE institutions rather than annihilated in a globalizing perspective.

An experiment on the parameter uncertainty of hydrological models with different levels of complexity in a climate change context

La possibilité d’estimer l’impact du changement climatique en cours sur le comportement hydrologique des hydro-systèmes est une nécessité pour anticiper les adaptations inévitables et nécessaires que doivent envisager nos sociétés. Dans ce contexte, ce projet doctoral présente une étude sur l’évaluation de la sensibilité des projections hydrologiques futures à : (i) La non-robustesse de l’identification des paramètres des modèles hydrologiques, (ii) l’utilisation de plusieurs jeux de paramètres équifinaux et (iii) l’utilisation de différentes structures de modèles hydrologiques. Pour quantifier l’impact de la première source d’incertitude sur les sorties des modèles, quatre sous-périodes climatiquement contrastées sont tout d’abord identifiées au sein des chroniques observées. Les modèles sont calés sur chacune de ces quatre périodes et les sorties engendrées sont analysées en calage et en validation en suivant les quatre configurations du Different Splitsample Tests (Klemeš, 1986;Wilby, 2005; Seiller et al. (2012);Refsgaard et al. (2014)). Afin d’étudier la seconde source d’incertitude liée à la structure du modèle, l’équifinalité des jeux de paramètres est ensuite prise en compte en considérant pour chaque type de calage les sorties associées à des jeux de paramètres équifinaux. Enfin, pour évaluer la troisième source d’incertitude, cinq modèles hydrologiques de différents niveaux de complexité sont appliqués (GR4J, MORDOR, HSAMI, SWAT et HYDROTEL) sur le bassin versant québécois de la rivière Au Saumon. Les trois sources d’incertitude sont évaluées à la fois dans conditions climatiques observées passées et dans les conditions climatiques futures. Les résultats montrent que, en tenant compte de la méthode d’évaluation suivie dans ce doctorat, l’utilisation de différents niveaux de complexité des modèles hydrologiques est la principale source de variabilité dans les projections de débits dans des conditions climatiques futures. Ceci est suivi par le manque de robustesse de l’identification des paramètres. Les projections hydrologiques générées par un ensemble de jeux de paramètres équifinaux sont proches de celles associées au jeu de paramètres optimal. Par conséquent, plus d’efforts devraient être investis dans l’amélioration de la robustesse des modèles pour les études d’impact sur le changement climatique, notamment en développant les structures des modèles plus appropriés et en proposant des procédures de calage qui augmentent leur robustesse. Ces travaux permettent d’apporter une réponse détaillée sur notre capacité à réaliser un diagnostic des impacts des changements climatiques sur les ressources hydriques du bassin Au Saumon et de proposer une démarche méthodologique originale d’analyse pouvant être directement appliquée ou adaptée à d’autres contextes hydro-climatiques.