Selective feeding centres in refugee settings: evaluation framework protocol

Selective feeding programs are centres for the treatment of persons suffering from acute malnutrition. Unlike chronic malnutrition, acute malnutrition reflects recent problems. In a crisis situation, wasting is preferred above other indicators because it is sensitive to rapid change, indicates present change, can be used to monitor the impact of interventions and is a good predictor of immediate mortality risk. This paper reviews the current approach being used in the field to evaluate the effectiveness of feeding programs. There is no comprehensive evaluation framework in place to assess the impact of feeding programs on mortality due to malnutrition. Some loose outcome measures, such as the number of children enrolled in a feeding centre, are being used to determine if a feeding centre should continue. In addition, malnutrition prevalence and crude mortality rates determined through nutritional and mortality surveys are used to assess the impact of feeding programs. This procedure does not take into account potential confounding factors that impact on malnutrition prevalence, including access to non-relief foods and the general food ration. Therefore, one could not confidently say that the reduction of malnutrition prevalence is a result of feeding programs. This paper presents an alternative approach to evaluating feeding centres.

An evaluation framework for the qualitative investigation of a physical activity prescription program in rural Victoria

Physical activity (PA) has consistently been shown to improve ones' physical, social and mental health. Hence, the rationale for promoting regular physical activity is relatively straightforward. However, what is less clear from the literature is the mix of strategies, (i.e. 'what is working and why?'), to increase in physical activity levels within a population. Of particular interest to primary care is the concept of Physical Activity Prescription Programs (PAPPs) delivered by general practitioners (GPs).

Several examples of a PAPP exist within England, America, New Zealand and Australia. These all aim to encourage GPs to deliver physical activity advice to their patients effectively and in a timely manner. Notwithstanding, the authors of a recent review of published literature on PAPPs, reveal a deficit of evidence regarding the components for successful PA interventions.

This research through an ethnographic enquiry aims to build on the evidence formulated to date. Through a case study research design, the researcher has developed a methodology to define what is/is not working within this recent trend.

In two rural Divisions of GP, participants have been identified as key stakeholders in the implementation of a PAPPs. They are categorised according to three theoretical paradigms, namely, Policy Makers, Linkers and Adaptors. Following this the three paradigms will be studied on the contextual factors, the characteristics and behaviours of members within all three paradigms. The study has also further defined certain elements for investigation, these include the:
intention of the players
effort undertaken by players, and
effect of parties within.

Primarily qualitative data will be collected; through Desk Analysis (Policies, Strategic and Business plans), Site Visits (Participant Observation) and semi-structured interviews. This presentation defines a qualitative framework and methodology for investigating the outcome of programs that historically has been evaluated using quantitative measures. Hence, the author of this study aims to present a qualitative investigation and subsequent results, defining aspects of a PAPP that allow for successful and sustainable implementation.

Using a knowledge management evaluation framework for improving an ERP system - a Hong Kong construction industry case study

Organisations need to rely on leadership, information support and human capital in order to ensure a knowledge advantage over their competitors. Knowledge management (KM) provides organisations with sustainable competitive advantage, because it becomes extremely difficult for an organisation to cut expenditure and increase revenue by simply reengineering its business model. Project delivery and success has been traditionally viewed and measured as management of a three-legged stool, with the legs defined as cost, schedule and quality. However, KM can be linked to success by organisations becoming more effective as well as being more efficient.

This paper uses a KM framework, the Knowledge Advantage (K-Adv), developed initially for use by construction organisations. It assesses the impact of leadership and its supporting information communication technology infrastructure on the ability of people (by effectively creating, sharing, disseminating and using knowledge) to facilitate sustainable competitive advantage.

A case study that is presented is based upon the experience of a leading construction company using an Enterprise Resources Planning (ERP) system to demonstrate the effectiveness of KM from a cost management business unit perspective. Results are evaluated using a capability maturity model (CMM) - that forms the core of the K-Adv tool - to help improve processes that meet the needs of the organisation operating in a highly dynamic business environment. The case study is part of a broader doctoral research project that uses action learning to facilitate and measure ERP improvement.

Multidimensional evaluation framework for virtual environment efficacy assessment

This research is related to the user-centred design and use of Virtual Environment (VE) training systems. A multidimensional user-centred systematic training evaluation framework that combines ideas from human-computer interaction, training, education and psychology was proposed, which contributes to better design and evaluation of VE user interfaces.

Evaluation framework underpinning the digital divas programme

In Australia, as elsewhere, women’s participation rates in Information Technology (IT) have been low. IT is the generic term used to refer to the many courses in the Computer Science, and Information Systems disciplines. While there have been a number of intervention programmes implemented aimed at encouraging women into IT and retaining them once there, few have included evaluations of the efficacy of the intervention. Thus little is known about the factors contributing to the success, or lack of success, of the interventions, or of the medium and longer term impacts for the participants. In this paper we briefly describe an intervention programme implemented with girls in the high school years. We present an evaluation framework providing a detailed overview of the aims and processes involved in the evaluation of the programme. Data for the evaluation were embedded within the data gathering methods associated with the research on the intervention itself.

Theorising about gender and computing interventions through an evaluation framework

Despite over 20 years of intervention programmes, the gender balance in the computing profession is not improving. It has been suggested that the problem with much of the research on gender and computing is that it is under-theorised. The contribution of this study is an evaluation framework, designed to evaluate gender and computing interventions, which will advance the theorisation of research through programme evaluation. This study was undertaken in three phases: The first involved theory building through an extensive review of the literature resulting in a conceptual framework for intervention programme evaluation. The second phase consisted of a multiple-case study of 14 major intervention programmes in Australia. Subsequent modifications to the conceptual framework resulted in the gender and computing intervention evaluation framework. The value of the framework was confirmed in phase three by intervention experts and showed that applying the framework will help programme champions to evaluate their programmes more thoroughly. The dissemination of sound evaluation results will then enable a deeper theorisation of the issues surrounding gender and computing interventions.

Evaluating impact of clinical guidelines using a realist evaluation framework

RATIONALE, AIMS AND OBJECTIVES: The Remote Primary Health Care Manuals (RPHCM) project team manages the development and publication of clinical protocols and procedures for primary care clinicians practicing in remote Australia. The Central Australian Rural Practitioners Association Standard Treatment Manual, the flagship manual of the RPHCM suite, has been evaluated for accessibility and acceptability in remote clinics three times in its 20-year history. These evaluations did not consider a theory-based framework or a programme theory, resulting in some limitations with the evaluation findings. With the RPHCM having an aim of enabling evidence-based practice in remote clinics and anecdotally reported to do so, testing this empirically for the full suite is vital for both stakeholders and future editions of the RPHCM. METHODS: The project team utilized a realist evaluation framework to assess how, why and for what the RPHCM were being used by remote practitioners. A theory regarding the circumstances in which the manuals have and have not enabled evidence-based practice in the remote clinical context was tested. The project assessed this theory for all the manuals in the RPHCM suite, across government and aboriginal community-controlled clinics, in three regions of Australia. CONCLUSION: Implementing a realist evaluation framework to generate robust findings in this context has required innovation in the evaluation design and adaptation by researchers. This article captures the RPHCM team's experience in designing this evaluation.

Automatic refinement of user requirements : a case study in software tool evaluation

This paper presents an assessment of system effectiveness in automatic requirements refinement by comparing results obtained from experts and novices with those achieved by the system. As the investigated system was a combination of a tightly inter-connected methods and a tool, the evaluation framework melded together a number of distinct methodological approaches structured into three empirical studies, which aimed at the construction of a case problem domain, calibrating the system using this defined domain elements and finally using the calibrated system to assess its effectiveness. In consequence, it was concluded that the evaluated methods and tools were effective in supporting requirements refinement.

Evaluating health-promoting schools in Hong Kong: development of a framework

Health-promoting schools (HPS)/healthy schools have existed internationally for about 15 years. Yet there are few comprehensive evaluation frameworks available which enable the outcomes of HPS initiatives to be assessed. This paper identifies an evaluation framework developed in Hong Kong. The framework uses a range of approaches to explore what schools actually do in their health promotion and health education initiatives. The framework, which is based on the WHO (Western Pacific Regional Office) Guidelines for HPS, is described in detail. The appropriate instruments for data collection are described and their origins identified. The evaluation plan and protocol, which underpinned the very comprehensive evaluation in Hong Kong, are explained. Finally, a case is argued for evaluation of HPS to be more in line with the educational dynamics of schools and the research literature on effective schooling, rather than focusing primarily on health-related measures.

Evaluation of nurses' perceptions of the impact of targeted depression education and a screening and referral tool in an acute cardiac setting.

AIMS AND OBJECTIVES: The aim of this study was to evaluate nurses' perceptions of an education programme and screening and referral tool designed for cardiac nurses to facilitate depression screening and referral procedures for patients with coronary heart disease. BACKGROUND: There is a high prevalence of depression in patients with coronary heart disease that is often undetected. It is important therefore that nurses working with cardiac patients are equipped with the knowledge and skills to recognise the signs and symptoms of depression and refer appropriately. DESIGN: A qualitative approach with purposive sampling and semi-structural interviews was implemented within the Donabedian 'Structure-Process-Outcome' evaluation framework. METHODS: Semi-structured interviews were conducted with 14 cardiac nurses working in a major metropolitan hospital six weeks post-attending an education programme on depression and coronary heart disease. Thematic data analysis was implemented, specifically adhering to Halcomb and Davidson's (2006) pragmatic data analysis, to examine nurse knowledge and experience of depression assessment and referral in an acute cardiac ward. RESULTS: The key findings of this study were that the education programme: (1) increased the knowledge base of nurses working with cardiac patients on comorbid depression and coronary heart disease, and (2) assisted in the identification of depression and the referral of 'at risk' patients. CONCLUSIONS: Emphasis was placed on the translational significance of educating cardiac nurses about depression via the introduction of a depression screening and referral instrument designed specifically for use in the cardiac ward. As a result, participants found they were better equipped to identify depressive symptoms and, guided by the screening instrument, to confidently instigate referral procedures. RELEVANCE TO CLINICAL PRACTICE: Much complexity lies in caring for cardiac patients with depression, including issues such as misdiagnosis. Targeted education, including use of appropriate instruments, has the potential to facilitate early recognition of the signs and symptoms of depression in the acute cardiac setting.

Realist evaluation of a remote hospital programme in the context of Australian health care reforms

Background: Since 2008, Australia has embarked on major healthcare reforms extending across all states and territories. Only limited evidence linking national healthcare reforms to improvement in public hospital performance exists. This evidence gap is even more pronounced in the case of remote hospital performance. This article describes a protocol retrospectively assessing a remote hospital programme to implement emergency department performance indicators, in the context of national reforms, over a period of 7 years (2008–2014). Challenges to implementing these reforms are explored.Method: Assessing the complex scenario of reform implementation requires an in-depth analysis, offered by a Realist Evaluation framework. Within this framework, a case study design is adopted to enable descriptive analysis. Interviews with key hospital stakeholders were followed by a literature review to identify a programme theory. The programme theory was articulated in the form of a preliminary context-mechanism-outcome configuration (CMOC). This theory will underlie further data collection, analysis, and interpretation. Both Realist Evaluation and case study allow flexibility in a choice of methods; both quantitative and qualitative methods will be incorporated. The thematic analysis will be employed to identify causal relationships and linkages in collected data.Discussion: Assembled data will be used to develop final CMOC patterns. The final CMOC will help in understanding the theory and mechanisms in use in the hospital.

Evaluating health promotion in schools: a case study of design, implementation and results from the Hong Kong Healthy Schools Award Scheme

Health promoting schools (HPS) and Healthy Schools Award Schemes from a number of countries have demonstrated positive changes in children’s health behaviours and the culture and organisation of the school. The Hong Kong Healthy Schools Award Scheme (HKHSA) aims to promote staff development, parental education, involvement of the whole school community, and linkage with different stakeholders to improve the health and well-being of the pupils, parents and staff, and the broader community, supported by a system to monitor the achievement. This concept is very much in line with the research literature on school effectiveness and improvement. The indicators examined to evaluate the success of the HKHSA reflect outcomes related to both health and education and are not limited to changes in population health status. The early results demonstrated significant improvements in various aspects of student health and also improvement in school culture and organisation. The evaluation framework described in this paper and data collected to assess how schools perform in the HKHSA scheme, provides insight into how HPSs could lead to better outcomes for both education and health.