PANA V : Evaluation of a Literacy Project

PANA V Evaluation of a Literacy ProjectSUMMARY AND CONCLUSIONSThis evaluation set out to explore the impact of the literacy work carried out through PANA V. It focussed on clarifying effects such as empowerment and poverty reduction in relation to the civil society. Two specific objectives were to evaluate the methodological approach and the didactic materials and to evaluate the sustainability of the project.Although the focus of the evaluation has been PANA V, the project has been evaluated in its context, as one in a series of five projects located in Rwanda ten years after the war and genocide. The conclusion will consider future plans in this field.The evaluator has striven to create a holistic picture of the effects of the project, although the given time for the evaluation was short. Only three weeks were spent in the field study and only ten days in the actual field. Although there were some organisational and logistic problems, as is common when carrying out a study like this in a poor country, many literacy sites were visited and quite many participators were interviewed. The overall impression from the study is overwhelmingly positive. So many people commit themselves in this task of teaching Rwandans reading, writing and numeracy. Despite harsh conditions learners strive to learn and group leaders devote themselves to the task. Many leaders on different levels try their very best to manage their difficult and demanding task. The main objective was to explore the impact of the project on poverty reduction, particularly on empowerment and strategies for everyday life. Women were to be regarded particularly. From the results it is clear that the project has a strong, positive impact both on poverty reduction and empowerment of marginalised groups. Among those who have benefited from the alphabetisation are mainly women. Unfortunately, when it comes to leaders in PANA, who may also be said to have benefited from the project, only a small minority is women. This is something that is recommended that it be reconsidered inside the organisation. As a majority of the targeted learners are women, and as the economic and social situation of women in Rwanda is generally weak, this is a question that I recommend the Pentesostal church and ADEPR to look particularly into. With many women being single breadwinners of their households, it is important that also women get access to positions that may bring benefits of different kind.It is also clear that the project has positive effects for the civil society. In the present situation in Rwanda, during the process of reconciliation and rapid progress, basic education for the poor majority is a democratic issue. In a country with a plethora of internet-cafés in the capital and a small minority that use cars and mobile-telephones to communicate nation-wide, it is of outmost importance that the majority acquires basic education, of which literacy is a central part. To strengthen the civil society in Rwanda literacy is important. One central issue is then that Rwanda develops toward becoming a country where literacy is used for the benefit of the citizens and it is a democratic issue that all citizens get an opportunity to participate. Crucial for this is that strong efforts are put into primary schools nation-wide. Literacy projects for adults, like PANA, may only complement these efforts, but they constitute important and necessary complements. Other relevant ways to promote literacy are campaigns in Radio and TV and through cultural events such as festivals, music and theatre. News papers, magazines and books are natural parts of such campaigns as well as adult education. As stated under the results not much can be said about the didactics in this evaluation. On the whole the methodology and the materials fill their function well and receive a high reputation. As people learn to read and write under very simple conditions, obviously the approach is appropriate. A few suggestions may be given from the study:•Focus groups leaders’ attention on clearness, that they show very clearly what is to be read. Good structuring is probably of great importance for many learners.•Make clear what is tested in the tests and consider the possibility to use a holistic test that would be more congruent with the methodology. The possibility to use only one grade, pass, would enable a more practical test, such as reading a short, relevant text, writing something relevant and solving practical mathematic problems. Avoid tests that demand school knowledge.•Avoid using methaphors such as “fight against illiteracy” and connections between illiteracy/literacy and darkness/light. It is not true that illiteracy causes bad things and that literacy only brings good. •Be prepared that it may be more difficult in the future to achieve the goals as it may be the case that the early learners where the ones who achieved easily. The goal of “literacy in six month” in PANA will probably hold only for some learners but also those who do not manage in six months need literacy skills.A third objective was to secure sustainability. As for sustainability of the project in itself, and of the literacy process, the main conclusion is that there is a good potential. The commitment and devotedness among many involved in PANA proves good. One weakness is individual leaders in ADEPR who do not see this as an important task for the Pentecostal church in Rwanda. Other weaknesses are the unwillingness to mention explicitly the wish, for example among group leaders, to get some kind of incentive and the fear of loosing believers by cooperation with other organisations. A higher degree of transparency in this issue would probably solve some irritations and tensions.As for the sustainability of the literacy skills much may be done to improve. The acquired skills seem to be comparably relevant. The level achieved, and the level tested, may be defined as basic literacy skills, consisting of basic reading, writing and numeracy skills. However, these skills are very restricted and there is a high risk that the skills will decline, which means that there is a high risk that people will forget how to read and write because of lack of exercising. From these conclusions a few suggestions for future development will be given.

Health system context and implementation of evidence-based practices-development and validation of the Context Assessment for Community Health (COACH) tool for low- and middle-income settings

Background: The gap between what is known and what is practiced results in health service users not benefitting from advances in healthcare, and in unnecessary costs. A supportive context is considered a key element for successful implementation of evidence-based practices (EBP). There were no tools available for the systematic mapping of aspects of organizational context influencing the implementation of EBPs in low- and middle-income countries (LMICs). Thus, this project aimed to develop and psychometrically validate a tool for this purpose. Methods: The development of the Context Assessment for Community Health (COACH) tool was premised on the context dimension in the Promoting Action on Research Implementation in Health Services framework, and is a derivative product of the Alberta Context Tool. Its development was undertaken in Bangladesh, Vietnam, Uganda, South Africa and Nicaragua in six phases: (1) defining dimensions and draft tool development, (2) content validity amongst in-country expert panels, (3) content validity amongst international experts, (4) response process validity, (5) translation and (6) evaluation of psychometric properties amongst 690 health workers in the five countries. Results: The tool was validated for use amongst physicians, nurse/midwives and community health workers. The six phases of development resulted in a good fit between the theoretical dimensions of the COACH tool and its psychometric properties. The tool has 49 items measuring eight aspects of context: Resources, Community engagement, Commitment to work, Informal payment, Leadership, Work culture, Monitoring services for action and Sources of knowledge. Conclusions: Aspects of organizational context that were identified as influencing the implementation of EBPs in high-income settings were also found to be relevant in LMICs. However, there were additional aspects of context of relevance in LMICs specifically Resources, Community engagement, Commitment to work and Informal payment. Use of the COACH tool will allow for systematic description of the local healthcare context prior implementing healthcare interventions to allow for tailoring implementation strategies or as part of the evaluation of implementing healthcare interventions and thus allow for deeper insights into the process of implementing EBPs in LMICs.

Knowledge translation in Uganda : a qualitative study of Ugandan midwives' and managers' perceived relevance of the sub-elements of the context cornerstone in the PARIHS framework

BACKGROUND: A large proportion of the annual 3.3 million neonatal deaths could be averted if there was a high uptake of basic evidence-based practices. In order to overcome this 'know-do' gap, there is an urgent need for in-depth understanding of knowledge translation (KT). A major factor to consider in the successful translation of knowledge into practice is the influence of organizational context. A theoretical framework highlighting this process is Promoting Action on Research Implementation in Health Services (PARIHS). However, research linked to this framework has almost exclusively been conducted in high-income countries. Therefore, the objective of this study was to examine the perceived relevance of the subelements of the organizational context cornerstone of the PARIHS framework, and also whether other factors in the organizational context were perceived to influence KT in a specific low-income setting. METHODS: This qualitative study was conducted in a district of Uganda, where focus group discussions and semi-structured interviews were conducted with midwives (n = 18) and managers (n = 5) within the catchment area of the general hospital. The interview guide was developed based on the context sub-elements in the PARIHS framework (receptive context, culture, leadership, and evaluation). Interviews were transcribed verbatim, followed by directed content analysis of the data. RESULTS: The sub-elements of organizational context in the PARIHS framework--i.e., receptive context, culture, leadership, and evaluation--also appear to be relevant in a low-income setting like Uganda, but there are additional factors to consider. Access to resources, commitment and informal payment, and community involvement were all perceived to play important roles for successful KT. CONCLUSIONS: In further development of the context assessment tool, assessing factors for successful implementation of evidence in low-income settings--resources, community involvement, and commitment and informal payment--should be considered for inclusion. For low-income settings, resources are of significant importance, and might be considered as a separate subelement of the PARIHS framework as a whole.