Scaling up community-based obesity prevention in Australia : background and evaluation design of the health promoting communities : being active eating well initiative

Background : There is only limited evidence available on how best to prevent childhood obesity and community-based interventions hold promise, as several successful interventions have now been published. The Victorian Government has recently funded six disadvantaged communities across Victoria, Australia for three years to promote healthy eating and physical activity for children, families, and adults in a community-based participatory manner. Five of these intervention communities are situated in Primary Care Partnerships and are the subject of this paper. The interventions will comprise a mixture of capacity-building, environmental, and whole-of-community approaches with targeted and population-level interventions. The specific intervention activities will be determined locally within each community through stakeholder and community consultation. Implementation of the interventions will occur through funded positions in primary care and local government. This paper describes the design of the evaluation of the five primary care partnership-based initiatives in the 'Go for your life' Health Promoting Communities: Being Active Eating Well (HPC:BAEW) initiative.

Methods/Design : A mixed method and multi-level evaluation of the HPC:BAEW initiative will capture process, impact and outcome data and involve both local and state-wide evaluators. There will be a combined analysis across the five community intervention projects with outcomes compared to a comparison group using a cross-sectional, quasi-experimental design. The evaluation will capture process, weight status, socio-demographic, obesity-related behavioral and environmental data in intervention and comparison areas. This will be achieved using document analysis, paper-based questionnaires, interviews and direct measures of weight, height and waist circumference from participants (children, adolescents and adults).

Discussion : This study will add significant evidence on how to prevent obesity at a population level in disadvantaged and ethnically diverse communities. The outcomes will have direct influence on policy and practice and guide the development and implementation of future obesity prevention efforts in Australia and internationally.

Process evaluation design and tools used in a kindergarten-based, family-involved intervention to prevent obesity in early childhood. The ToyBox-study

Process evaluation (PE) is used for the in-depth evaluation of the implementation process of health promotion programmes. The aim of the current paper was to present the PE design and tools used in the ToyBox-intervention. The PE design was based on a three-step approach, including the identification of ToyBox-specific PE elements (step 1), the development of PE tools and harmonization of procedures (step 2), and the implementation of PE using standardized protocol and tools across the intervention countries (step 3). Specifically, to evaluate the implementation of the intervention, teachers' monthly logbooks were recorded (dose delivered, fidelity, dose received); post-intervention questionnaires were completed by parents/caregivers and teachers (dose received); participation and attrition rates were recorded (recruitment, reach); and audit questionnaires and retrospective information on weather conditions were collected (physical and social environment within which the intervention was implemented). Regarding the teachers' training sessions, the researchers who performed the trainings completed evaluation forms and documented teachers' attendance after each training (dose delivered, fidelity, dose received) and teachers completed evaluation forms after each training (dose received). The PE performed in the ToyBox-intervention may contribute in the evaluation of its effectiveness, guide the revision of the intervention material and provide insights for future health promotion programmes and public health policy.

Addressing Multilevel Program Complexity by Evaluation Design

In this paper, we present the evaluation design for a complex multilevel program recently introduced in Switzerland. The evaluation embraces the federal level, the cantonal program level, and the project level where target groups are directly addressed. We employ Pawson and Tilley’s realist evaluation approach, in order to do justice to the varying context factors that impact the cantonal programs leading to varying effectiveness of the implemented activities. The application of the model to the canton of Uri shows that the numerous vertical and horizontal relations play a crucial role for the program’s effectiveness. As a general learning for the evaluation of complex programs, we state that there is a need to consider all affected levels of a program and that no monocausal effects can be singled out in programs where multiple interventions address the same problem. Moreover, considering all affected levels of a program can mean going beyond the borders of the actual program organization and including factors that do not directly interfere with the policy delivery as such. In particular, we found that the relationship between the cantonal and the federal level was a crucial organizational factor influencing the effectiveness of the cantonal program.

A case study on a design-informed developmental evaluation

Developmental evaluation (DE) is an evaluation approach that aims to support the development of an innovation (Patton, 1994, 2011). This aim is achieved through supporting clients’ information needs through evaluative inquiry as they work to develop and refine the innovation. While core concepts and principles are beginning to be articulated and refined, challenges remain as to how to focus a developmental evaluation beyond those knowledge frameworks most immediate to clients to support innovation development. Anchoring a DE in knowledge frameworks other than those of the clients might direct attention to issues not yet obvious to clients, but which might further the goal of supporting innovation development if attended to. Drawing concepts and practices from the field of design may be one avenue with which to inform developmental evaluation in achieving its aim. Through a case study methodology, this research seeks to understand the nuances of operationalizing the guiding principles of DE as well as to investigate the utility, feasibility, and consequences of integrating design concepts and practices into developmental evaluation (design-informed developmental evaluation, “DI-DE”). It does so by documenting the efforts of a design-informed developmental evaluator and a task force of educators and researchers in a Faculty of Education as they work to develop a graduate-level education program. A systematic review into those purposeful efforts made to introduce DI-DE thinking into task force deliberations, and an analysis into the responses and consequences of those efforts shed light on what it had meant to practice DI-DE. As a whole, this research on evaluation is intended to further contemporary thinking about the closely coupled relationship between program development and evaluation in complex and dynamic environments.

Problem solving in a middle school robotics design classroom

Little research has been conducted on how students work when they are required to plan, build and evaluate artefacts in technology rich learning environments such as those supported by tools including flow charts, Labview programming and Lego construction. In this study, activity theory was used as an analytic tool to examine the social construction of meaning. There was a focus on the effect of teachers’ goals and the rules they enacted upon student use of the flow chart planning tool, and the tools of the programming language Labview and Lego construction. It was found that the articulation of a teacher’s goals via rules and divisions of labour helped to form distinct communities of learning and influenced the development of different problem solving strategies. The use of the planning tool flow charting was associated with continuity of approach, integration of problem solutions including appreciation of the nexus between construction and programming, and greater educational transformation. Students who flow charted defined problems in a more holistic way and demonstrated more methodical, insightful and integrated approaches to their use of tools. The findings have implications for teaching in design dominated learning environments.

Design of open source framework for traffic and travel simulation

For the evaluation, design, and planning of traffic facilities and measures, traffic simulation packages are the de facto tools for consultants, policy makers, and researchers. However, the available commercial simulation packages do not always offer the desired work flow and flexibility for academic research. In many cases, researchers resort to designing and building their own dedicated models, without an intrinsic incentive (or the practical means) to make the results available in the public domain. To make matters worse, a substantial part of these efforts pertains to rebuilding basic functionality and, in many respects, reinventing the wheel. This problem not only affects the research community but adversely affects the entire traffic simulation community and frustrates the development of traffic simulation in general. For this problem to be addressed, this paper describes an open source approach, OpenTraffic, which is being developed as a collaborative effort between the Queensland University of Technology, Australia; the National Institute of Informatics, Tokyo; and the Technical University of Delft, the Netherlands. The OpenTraffic simulation framework enables academies from geographic areas and disciplines within the traffic domain to work together and contribute to a specific topic of interest, ranging from travel choice behavior to car following, and from response to intelligent transportation systems to activity planning. The modular approach enables users of the software to focus on their area of interest, whereas other functional modules can be regarded as black boxes. Specific attention is paid to a standardization of data inputs and outputs for traffic simulations. Such standardization will allow the sharing of data with many existing commercial simulation packages.

Improving the design of online applications for social benefit through a behaviour change model

Community support agencies routinely employ a web presence to provide information on their services. While this online information provision helps to increase an agency’s reach, this paper argues that it can be further extended by mapping relationships between services and by facilitating two-way communication and collaboration with local communities. We argue that emergent technologies, such as locative media and networking tools, can assist in harnessing this social capital. However, new applications must be designed in ways that both persuade and support community members to contribute information and support others in need. An analysis of the online presence of community service agencies and social benefit applications is presented against Fogg’s Behaviour Model. From this evaluation, design principles are proposed for developing new locative, collaborative online applications for social benefit.

Course evaluation : reconfigurations for learning with learning management systems

The introduction of online delivery platforms such as learning management systems (LMS) in tertiary education has changed the methods and modes of curriculum delivery and communication. While course evaluation methods have also changed from paper-based in-class-administered methods to largely online-administered methods, the data collection instruments have remained unchanged. This paper reports on a small exploratory study of two tertiary-level courses. The study investigated why design of the instruments and methods to administer surveys in the courses are ineffective measures against the intrinsic characteristics of online learning. It reviewed the students' response rates of the conventional evaluations for the courses over an eight-year period. It then compared a newly developed online evaluation and the conventional methods over a two-year period. The results showed the response rates with the new evaluation method increased by more than 80% from the average of the conventional evaluations (below 30%), and the students' written feedback was more detailed and comprehensive than in the conventional evaluations. The study demonstrated the possibility that the LMS-based learning evaluation can be effective and efficient in terms of the quality of students' participation and engagement in their learning, and for an integrated pedagogical approach in an online learning environment.

ITSM ProcessGuide – a longitudinal and multi-method field study for real-world DSR artifact evaluation

Process guidance supports users to increase their process model understanding, process execution effectiveness as well as efficiency, and process compliance performance. This paper presents a research in progress encompassing our ongoing DSR project on Process Guidance Systems and a field evaluation of the resulting artifact in cooperation with a company. Building on three theory-grounded design principles, a Process Guidance System artifact for the company’s IT service ticketing process is developed, deployed and used. Fol-lowing a multi-method approach, we plan to evaluate the artifact in a longitudinal field study. Thereby, we will not only gather self-reported but also real usage data. This article describes the development of the artifact and discusses an innovative evaluation approach.

Environmental indicator frameworks to design and assess environmental monitoring programs

Impact Assessment and Project Appraisal, vol. 22, n.1, March 2004, p. 47–62

Evaluation of the agricultural vocational training programmes conducted by the Krishi Vigyan Kendras (Farm Science Centres) in Indian Punjab

Krishin Vigyan Kendras-KVKs (Farm Science Centres) have been established by the Indian Council of Agricultural Research in 569 districts. The trust areas of KVKs are refinement and demonstration of technologies, and training of farmers and extension functionaries. Imparting vocational trainings in agriculture and allied fields for the rural youth is one of its mandates. The study was undertaken to do a formative and summative (outcome and impact) evaluation of the beekeeping and mushroom growing vocational training programmes in the Indian state of Punjab. One-group pre and post evaluation design was employed for conducting a formative and outcome evaluation. The knowledge tests were administered to 35 beekeeping and 25 mushroom cultivation trainees, before and after the training programmes organized in 2004. The trainees significantly gained in knowledge. A separate sample of 640 trainees, trained prior to 2004, was selected for finding the adoption status. Out of 640, a sample of 200 was selected by proportionate sampling technique out of three categories, namely: non-adopters, discontinued-adopters and continued-adopters for evaluating the long-term impact of these training programmes. Ex-post-facto one-shot case study design was applied for this impact analysis. The vocational training programmes have resulted in continued-adoption of beekeeping and mushroom cultivation enterprises by 20% and 51% trained farmers, respectively. Age and trainee occupation had significant influence on the adoption decision of beekeeping vocation, whereas education and family income significantly affected the adoption decision of mushroom cultivation. The continued adopters of beekeeping and mushroom growing had increased their family income by 49% and 24%, respectively. These training programmes are augmenting the dwindling farm income of the farmers in Indian Punjab.

Evaluation of the childhood obesity prevention program kids - go for your life

Background
Kids - 'Go for your life' (K-GFYL) is an award-based health promotion program being implemented across Victoria, Australia. The program aims to reduce the risk of childhood obesity by improving the socio-cultural, policy and physical environments in children's care and educational settings. Membership of the K-GFYL program is open to all primary and pre-schools and early childhood services across the State. Once in the program, member schools and services are centrally supported to undertake the health promotion (intervention) activities. Once the K-GFYL program 'criteria' are reached the school/service is assessed and 'awarded'. This paper describes the design of the evaluation of the statewide K-GFYL intervention program.

Methods/Design
The evaluation is mixed method and cross sectional and aims to:
1) Determine if K-GFYL award status is associated with more health promoting environments in schools/services compared to those who are members only;
2) Determine if children attending K-GFYL award schools/services have higher levels of healthy eating and physical activity-related behaviors compared to those who are members only;
3) Examine the barriers to implementing and achieving the K-GFYL award; and
4) Determine the economic cost of implementing K-GFYL in primary schools
Parent surveys will capture information about the home environment and child dietary and physical activity-related behaviors. Environmental questionnaires in early childhood settings and schools will capture information on the physical activity and nutrition environment and current health promotion activities. Lunchbox surveys and a set of open-ended questions for kindergarten parents will provide additional data. Resource use associated with the intervention activities will be collected from primary schools for cost analysis.

Discussion
The K-GFYL award program is a community-wide intervention that requires a comprehensive, multi-level evaluation. The evaluation design is constrained by the lack of a non-K-GFYL control group, short time frames and delayed funding of this large scale evaluation across all intervention settings. However, despite this, the evaluation will generate valuable evidence about the utility of a community-wide environmental approach to preventing childhood obesity which will inform future public health policies and health promotion programs internationally.

Trial Registration
ACTRN12609001075279