Evaluation use within a humanitarian non-governmental organization’s health care user-fee exemption program in West Africa

L’évaluation de l’action humanitaire (ÉAH) est un outil valorisé pour soutenir l’imputabilité, la transparence et l’efficience de programmes humanitaires contribuant à diminuer les inéquités et à promouvoir la santé mondiale. L’EAH est incontournable pour les parties prenantes de programme, les bailleurs de fonds, décideurs et intervenants souhaitant intégrer les données probantes aux pratiques et à la prise de décisions. Cependant, l’utilisation de l’évaluation (UÉ) reste incertaine, l’ÉAH étant fréquemment menée, mais inutilisé. Aussi, les conditions influençant l’UÉ varient selon les contextes et leur présence et applicabilité au sein d’organisations non-gouvernementales (ONG) humanitaires restent peu documentées. Les évaluateurs, parties prenantes et décideurs en contexte humanitaire souhaitant assurer l’UÉ pérenne détiennent peu de repères puisque rares sont les études examinant l’UÉ et ses conditions à long terme. La présente thèse tend à clarifier ces enjeux en documentant sur une période de deux ans l’UÉ et les conditions qui la détermine, au sein d’une stratégie d’évaluation intégrée au programme d’exemption de paiement des soins de santé d’une ONG humanitaire. L’objectif de ce programme est de faciliter l’accès à la santé aux mères, aux enfants de moins de cinq ans et aux indigents de districts sanitaires au Niger et au Burkina Faso, régions du Sahel où des crises alimentaires et économiques ont engendré des taux élevés de malnutrition, de morbidité et de mortalité. Une première évaluation du programme d’exemption au Niger a mené au développement de la stratégie d’évaluation intégrée à ce même programme au Burkina Faso. La thèse se compose de trois articles. Le premier présente une étude d’évaluabilité, étape préliminaire à la thèse et permettant de juger de sa faisabilité. Les résultats démontrent une logique cohérente et plausible de la stratégie d’évaluation, l’accessibilité de données et l’utilité d’étudier l’UÉ par l’ONG. Le second article documente l’UÉ des parties prenantes de la stratégie et comment celle-ci servit le programme d’exemption. L’utilisation des résultats fut instrumentale, conceptuelle et persuasive, alors que l’utilisation des processus ne fut qu’instrumentale et conceptuelle. Le troisième article documente les conditions qui, selon les parties prenantes, ont progressivement influencé l’UÉ. L’attitude des utilisateurs, les relations et communications interpersonnelles et l’habileté des évaluateurs à mener et à partager les connaissances adaptées aux besoins des utilisateurs furent les conditions clés liées à l’UÉ. La thèse contribue à l’avancement des connaissances sur l’UÉ en milieu humanitaire et apporte des recommandations aux parties prenantes de l’ONG.

An evaluation framework for educational reform projects for teacher quality improvements in developing countries : a case study of Egyptian education reform

The role of the evaluation for Official Development Assistance (ODA) enterprises including educational development has become critical after increasing “aid fatigue” experienced by the international community in the 1990s. To date, however, monitoring and evaluating outcomes of the projects has been limited to the project life. Consequently these have been mainly through the international aid agencies. Furthermore, the monitoring and evaluation led by international aid agencies have paid little attention to aspects of the sustainability of technical cooperation in educational development. To sustain the impact of technical cooperation, the reinforcement of evaluation has drawn increasing attention in light of the emerging modalities in international development. Therefore this research was inspired to investigate alternative evaluation frameworks for an educational reform project for teacher quality improvement that may increase possibilities for long term sustainability. Importantly, the new modalities in international development and educational issues provide new options. In addition, the research reviewed theoretical and practical issues surrounding evaluation in general, and highlighted the evaluation of education reform projects. The research reported explored via case studies, the evaluation processes employed by the Egyptian education reform projects implemented by the Japan International Cooperation Agency (JICA) and the United Nations Children’s Fund (UNICEF). The case studies used three data sources (archival and relevant documents, a survey questionnaire and interviews) to illuminate the contextually-embedded evaluation processes. The research found that process evaluation is a potential alternative method since it is likely to be locally institutionalised, which may yield long-term sustainability of the projects.

Comprehensive evaluation of driver gaze pattern using fuzzy rules for driver training

The over representation of novice drivers in crashes is alarming. Research indicates that one in five drivers’ crashes within their first year of driving. Driver training is one of the interventions aimed at decreasing the number of crashes that involve young drivers. Currently, there is a need to develop comprehensive driver evaluation system that benefits from the advances in Driver Assistance Systems. Since driving is dependent on fuzzy inputs from the driver (i.e. approximate distance calculation from the other vehicles, approximate assumption of the other vehicle speed), it is necessary that the evaluation system is based on criteria and rules that handles uncertain and fuzzy characteristics of the drive. This paper presents a system that evaluates the data stream acquired from multiple in-vehicle sensors (acquired from Driver Vehicle Environment-DVE) using fuzzy rules and classifies the driving manoeuvres (i.e. overtake, lane change and turn) as low risk or high risk. The fuzzy rules use parameters such as following distance, frequency of mirror checks, gaze depth and scan area, distance with respect to lanes and excessive acceleration or braking during the manoeuvre to assess risk. The fuzzy rules to estimate risk are designed after analysing the selected driving manoeuvres performed by driver trainers. This paper focuses mainly on the difference in gaze pattern for experienced and novice drivers during the selected manoeuvres. Using this system, trainers of novice drivers would be able to empirically evaluate and give feedback to the novice drivers regarding their driving behaviour.

An assessment system for evaluation of driving competencies

Automobiles have deeply impacted the way in which we travel but they have also contributed to many deaths and injury due to crashes. A number of reasons for these crashes have been pointed out by researchers. Inexperience has been identified as a contributing factor to road crashes. Driver’s driving abilities also play a vital role in judging the road environment and reacting in-time to avoid any possible collision. Therefore driver’s perceptual and motor skills remain the key factors impacting on road safety. Our failure to understand what is really important for learners, in terms of competent driving, is one of the many challenges for building better training programs. Driver training is one of the interventions aimed at decreasing the number of crashes that involve young drivers. Currently, there is a need to develop comprehensive driver evaluation system that benefits from the advances in Driver Assistance Systems. A multidisciplinary approach is necessary to explain how driving abilities evolves with on-road driving experience. To our knowledge, driver assistance systems have never been comprehensively used in a driver training context to assess the safety aspect of driving. The aim and novelty of this thesis is to develop and evaluate an Intelligent Driver Training System (IDTS) as an automated assessment tool that will help drivers and their trainers to comprehensively view complex driving manoeuvres and potentially provide effective feedback by post processing the data recorded during driving. This system is designed to help driver trainers to accurately evaluate driver performance and has the potential to provide valuable feedback to the drivers. Since driving is dependent on fuzzy inputs from the driver (i.e. approximate distance calculation from the other vehicles, approximate assumption of the other vehicle speed), it is necessary that the evaluation system is based on criteria and rules that handles uncertain and fuzzy characteristics of the driving tasks. Therefore, the proposed IDTS utilizes fuzzy set theory for the assessment of driver performance. The proposed research program focuses on integrating the multi-sensory information acquired from the vehicle, driver and environment to assess driving competencies. After information acquisition, the current research focuses on automated segmentation of the selected manoeuvres from the driving scenario. This leads to the creation of a model that determines a “competency” criterion through the driving performance protocol used by driver trainers (i.e. expert knowledge) to assess drivers. This is achieved by comprehensively evaluating and assessing the data stream acquired from multiple in-vehicle sensors using fuzzy rules and classifying the driving manoeuvres (i.e. overtake, lane change, T-crossing and turn) between low and high competency. The fuzzy rules use parameters such as following distance, gaze depth and scan area, distance with respect to lanes and excessive acceleration or braking during the manoeuvres to assess competency. These rules that identify driving competency were initially designed with the help of expert’s knowledge (i.e. driver trainers). In-order to fine tune these rules and the parameters that define these rules, a driving experiment was conducted to identify the empirical differences between novice and experienced drivers. The results from the driving experiment indicated that significant differences existed between novice and experienced driver, in terms of their gaze pattern and duration, speed, stop time at the T-crossing, lane keeping and the time spent in lanes while performing the selected manoeuvres. These differences were used to refine the fuzzy membership functions and rules that govern the assessments of the driving tasks. Next, this research focused on providing an integrated visual assessment interface to both driver trainers and their trainees. By providing a rich set of interactive graphical interfaces, displaying information about the driving tasks, Intelligent Driver Training System (IDTS) visualisation module has the potential to give empirical feedback to its users. Lastly, the validation of the IDTS system’s assessment was conducted by comparing IDTS objective assessments, for the driving experiment, with the subjective assessments of the driver trainers for particular manoeuvres. Results show that not only IDTS was able to match the subjective assessments made by driver trainers during the driving experiment but also identified some additional driving manoeuvres performed in low competency that were not identified by the driver trainers due to increased mental workload of trainers when assessing multiple variables that constitute driving. The validation of IDTS emphasized the need for an automated assessment tool that can segment the manoeuvres from the driving scenario, further investigate the variables within that manoeuvre to determine the manoeuvre’s competency and provide integrated visualisation regarding the manoeuvre to its users (i.e. trainers and trainees). Through analysis and validation it was shown that IDTS is a useful assistance tool for driver trainers to empirically assess and potentially provide feedback regarding the manoeuvres undertaken by the drivers.

Framing local stories for community consumption facilitation and evaluation in narrative driven arts-based projects

Cyclone Yasi struck the Cassowary Coast of Queensland in the early hours of Feb 3, 2011, destroying many homes sand property, including the destruction of the Cardwell and district historical society’s premises. With their own homes flattened, many were forced to live in mobile accommodation, with extended family, or leave altogether. The historical society members however were more devastated by their flattened foreshore museum and loss of their collection material. A call for assistance was made through the OHAA Qld branch, who along with QUT sponsored a trip to somehow plan how they could start to pick up the pieces to start again. This presentation highlights the need for communities to gather, preserve and present their own stories, in a way that is sustainable and meaningful to them, but that good advice and support along the way is important. Two 2 day workshops were held in March and then September, augmented by plenty of email correspondence and phone calls in between. Participants learnt that if they could conduct quality oral history interviews, they could later use these in many exhibitable ways including: documentary pieces; digital stories; photographic collections; creative short stories; audio segments –while also drawing closely together a suffering community. This story is not only about the people who were interviewed about the night Yasi struck, but the amazing women (all over 50) of the historical society who were willing to try and leap the digital divide that faces older Australians, especially those in rural Australia, so that their older local stories would not be lost and so that new stories could also be remembered.

Facilitating local stories in post-disaster regional communities : evaluation in narrative-driven oral history projects

Cyclone Yasi struck the Cassowary Coast of Northern Queensland, Australia, in the early hours of February 3, 2011, destroying many homes and property, including the destruction of the Cardwell and district historical society’s premises. With their own homes flattened, many residents were forced to live in mobile accommodation, with extended family, or leave the area altogether. The historical society members seemed, however, particularly devastated by their flattened foreshore museum and loss of their precious collection of material. A call for assistance was made through the Oral History Association of Australia’s Queensland branch (OHAA Qld), which along with a Queensland University of Technology (QUT) research team sponsored a trip to best plan how they could start to pick up the pieces to rebuild the museum. This chapter highlights the need for communities to gather, preserve and present their own stories, in a way that is sustainable and meaningful to them – whether that be because of a disaster, or as they go about life in their contemporary communities – the key being that good advice, professional support and embedded evaluation practices at crucial moments along the way can be critically important.

Evaluating NEXUS: A suicide prevention program for young people from refugee backgrounds (2007 – 2009)

Although suicide deaths in Australia continue to decline since the peak of 2,720 suicide deaths in 1997, youth suicide and self-harm are a major health issue. In 2006, in the 20 to 24 year age group, suicide accounted for approximately 21% of all male deaths and 14% of all female deaths. There is a lack of solid data on the rates of suicide and self-harm among young people from refugee backgrounds. However, this population faces a significant number of post-resettlement stressors that may add to their vulnerability and increase their risk of suicide and self-harm. The NEXUS program is an innovative strategy developed by the Queensland Program of Assistance to Survivors of Torture and Trauma (QPASTT) that aims to reduce risk factors for suicide and self-harm and to promote protective factors among youth from refugee backgrounds living in Brisbane and Toowoomba. QPASTT is a non-government organisation that provides culturally appropriate support services to refugee and humanitarian entrants to Australia. QPASTT’s primary function is to provide counselling, advocacy support and community development activities for survivors of torture and trauma at an individual, family and community level. Since 2002 the NEXUS program has been developed and implemented by QPASTT. Since then, this multi-component program has been funded by the Commonwealth Department of Health and Ageing through the National Suicide Prevention Strategy (NSPS). NSPS funding of local community suicide prevention activities will contribute to the outcomes specified in the strategic framework: Living is for Everyone (LIFE): A framework for prevention of suicide and self-harm in Australia. The focus of this report is the evaluation of the NEXUS program conducted by QPASTT between August 2007 and May 2009.

Spotlight on animating spaces: Evaluation of Artslink Queensland's Animating Spaces 2013 & 2014

Animating Spaces 2013-14 Interim Report Artslink Queensland’s Animating Spaces initiative is a three-year statewide project (2013-2015) that aims to revitalise and celebrate non-traditional, significant, and unusual spaces within fifteen regional Queensland communities. After the completion of two years of Animating Spaces this is the first public interim report highlighting outcomes to date. A final evaluation report will be collated in early 2016 that will capture the Animating Spaces project in its entirety. Developed by Queensland University of Technology evaluation team, Professor Helen Klaebe and Dr. Elizabeth Ellison with the extraordinary assistance of Artslink Queensland staff, in particular Kerryanne Farrer.

Assistance in poverty alleviation through improved aquatic resources management in Asia-Pacific: DoF/NACA-STREAM/FAO Workshop on livelihoods approaches and analysis. Yangon, Union of Myanmar, 11-15 May 2004

This is the report of the “DoF/NACA-STREAM/FAO Workshop on Livelihoods Approaches and Analysis” that was conducted in Yangon, Union of Myanmar from 11-15 May 2004. The purpose of the workshop was to develop and document mechanisms for training in livelihoods approaches and analysis, and to build national capacity to conduct livelihoods studies. The workshop in Yangon was the first STREAM event in Myanmar, with colleagues coming to participate from Yangon and many Divisions and States throughout the country. The workshop in Yangon was the fourth in a series, the first of which was held in Iloilo City, Philippines, in November 2003, the second in Ranchi, India, in February 2004, and the third in Vientiane, Lao PDR in March 2004. A subsequent workshop will take place in Yunnan, China. The objectives of the workshop were to: Understand issues of interest to people whose livelihoods include aquatic resources management, especially those with limited resources Build “(national) livelihoods teams” to do livelihoods analyses and training, and share their experiences with communities and other stakeholders Share understandings of livelihoods approaches and analysis using participatory methods Review current NACA-STREAM livelihoods analysis documentation, adapt and supplement, towards the drafting of a Guide for Livelihoods Analysis Experience the use of participatory tools for livelihoods analysis Plan activities for carrying out livelihoods analyses, and Consider how to build capacity in monitoring and evaluation (M&E) and “significant change”. (Pdf contains 56 pages).

Evaluation of woman fisherfolks involvement in fisheries activities in Kainji Lake basin

The extent of rural women fisherfolks contribution to fisheries was grossly under estimated and certainly under valued. The basis of their involvement in fisheries activities is to make themselves equal partners to men productive and self reliant participants in the process of improving their own and their family living standard and to enable them realize their full potentials. This survey is informed a system of data collection with the aid of questionnaires and analysis. Ninety-six of the questionnaires were administered to women fisher folks in eight fishing communities selected at random. Thirty seven (97) percent of the women fisher folks are between the ages of 25-35 years, 59.4% of them can only read and write in Arabic language and 21.9% only are literature in western education, which is a general characteristic of the rural population in Nigeria. 24.0% of the respondents are in full time fishing activities while 65-5% are fishmongers. They belong to cooperative societies but only 26% indicated to have benefited from loan and credit facilities. 84.4% of the respondents are sustained by the business. The major problems facing women fisherfolks includes poor market price, fish spoilage, high transport cost and lack of access to loan and credit facilities and extension assistance. Solution to these problems will increase their status benefit and development

Qualitative evaluation of KA24 (Knowledge Access 24)

Durbin, J. & Urquhart, C. (2003). Qualitative evaluation of KA24 (Knowledge Access 24). Aberystwyth: Department of Information Studies, University of Wales Aberystwyth. Sponsorship: Knowledge Access 24 (NHS)

Multiple Response System: Evaluation of Policy Change in North Carolina's Child Welfare System.

Systemic challenges within child welfare have prompted many states to explore new strategies aimed at protecting children while meeting the needs of families, but doing so within the confines of shrinking budgets. Differential Response has emerged as a promising practice for low or moderate risk cases of child maltreatment. This mixed methods evaluation explored various aspects of North Carolina's differential response system, known as the Multiple Response System (MRS), including: child safety, timeliness of response and case decision, frontloading of services, case distribution, implementation of Child and Family Teams, collaboration with community-based service providers and Shared Parenting. Utilizing Child Protective Services (CPS) administrative data, researchers found that compared to matched control counties, MRS: had a positive impact on child safety evidenced by a decline in the rates of substantiations and re-assessments; temporarily disrupted timeliness of response in pilot counties but had no effect on time to case decision; and increased the number of upfront services provided to families during assessment. Qualitative data collected through focus groups with providers and phone interviews with families provided important information on key MRS strategies, highlighting aspects that families and social workers like as well as identifying areas for improvement. This information is useful for continuous quality improvement efforts, particularly related to the development of training and technical assistance programs at the state and local level.

Limitaciones y obstáculos de MINUSTAH en materia de cooperación humanitaria y desarrollo en Haití (2000-2014)

Este estudio de caso pretende analizar las limitaciones y obstáculos a los que la misión de estabilización, MINUSTAH, se ha enfrentado en el proceso de reconstrucción de Haití antes, durante y después del terremoto de 2010. Además, busca identificar los elementos que han hecho de MINUSTAH una misión cuyo objetivo inicial de carácter político y estabilización se orientó hacia la asistencia humanitaria a causa de una catástrofe natural tal como el terremoto de 2010. En esta medida y bajo el marco de cooperación internacional para el desarrollo, este estudio de caso avanza por un proceso evaluativo del papel de MINUSTAH en la consolidación del Estado haitiano.

Evaluation of reanalysis rainfall estimates over Ethiopia

There is a pressing need for good rainfall data for the African continent both for humanitarian and climatological purposes. Given the sparseness of ground-based observations, one source of rainfall information is Numerical Weather Prediction (NWP) model outputs. The aim of this article is to investigate the quality of two NWP products using Ethiopia as a test case. The two products evaluated are the ERA-40 and NCEP reanalysis rainfall products. Spatial, seasonal and interannual variability of rainfall have been evaluated for Kiremt (JJAS) and Belg (FMAM) seasons at a spatial scale that reflects the local variability of the rainfall climate using a method which makes optimum use of sparse gauge validation data. We found that the spatial pattern of the rainfall climatology is captured well by both models especially for the main rainy season Kiremt. However, both models tend to overestimate the mean rainfall in the northwest, west and central regions but underestimate in the south and east. The overestimation is greater for NCEP in Belg season and greater for ERA-40 in Kiremt Season. ERA-40 captures the annual cycle over most of the country better than NCEP, but strongly exaggerates the Kiremt peak in the northwest and west. The overestimation in Kiremt appears to have been reduced since the assimilation of satellite data increased around 1990. For both models the interannual variability is less well captured than the spatial and seasonal variability. Copyright © 2008 Royal Meteorological Society

Uptake pathways: the potential of Bayesian belief networks to assist the management, monitoring and evaluation of development-orientated research

The effectiveness of development assistance has come under renewed scrutiny in recent years. In an era of growing economic liberalisation, research organisations are increasingly being asked to account for the use of public funds by demonstrating achievements. However, in the natural resources (NR) research field, conventional economic assessment techniques have focused on quantifying the impact achieved rather understanding the process that delivered it. As a result, they provide limited guidance for planners and researchers charged with selecting and implementing future research. In response, “pathways” or logic models have attracted increased interest in recent years as a remedy to this shortcoming. However, as commonly applied these suffer from two key limitations in their ability to incorporate risk and assess variance from plan. The paper reports the results of a case study that used a Bayesian belief network approach to address these limitations and outlines its potential value as a tool to assist the planning, monitoring and evaluation of development-orientated research.