Meta-evaluation to improve learning, evaluation capacity development and sustainability : Findings from a participatory evaluation project in Nepal

Recent studies suggest that meta-evaluation can be valuable in developing new approaches to evaluation, building evaluation capacities, and enhancing organizational learning. These new extensions of the concept of meta-evaluation are significant, given the growing emphasis on improving the quality and effectiveness of evaluation practices in the South Asian region. Following a review of the literature, this paper presents a case study of the use of concurrent meta-evaluation in the four-year project Assessing Communication for Social Change which developed and trialled a participatory impact assessment methodology in collaboration with a development communication Non-government organization (NGO) in Nepal. Key objectives of the meta-evaluation included to: continuously develop, adapt and improve the impact assessment methodology, Monitoring and Evaluation (M&E) systems and process and other project activities; identify impacts of the project; and build capacities in critical reflection and review. Our analysis indicates that this meta-evaluation was essential to understanding various constraints related to the organizational context that affected the success of the project and the development of improved M&E systems and capacities within the NGO. We identified several limitations of our meta-evaluation methods, which were balanced by the strengths of other methods. Our case study suggests that as well as assessing the quality, credibility and value of evaluation practices, meta-evaluations need to focus on important contextual issues that can have significant impacts on the outcomes of participatory evaluation projects. They include hierarchical organizational cultures, communication barriers, power/knowledge relations, and the time and resources available. Meta-evaluations also need to consider wider issues such as the sustainability of evaluation systems and approaches.

Reconceptualising policy integration in road safety management

Unless sustained, coordinated action is generated in road safety, road traffic deaths are poised to rise from approximately 1.3 to 1.9 million a year by 2020 (Krug, 2012). To generate this harmonised response, road safety management agencies are being urged to adopt multisectoral collaboration (WHO, 2009b), which is achievable through the principle of policy integration. Yet policy integration, in its current hierarchical format, is marred by a lack of universality of its interpretation, a failure to anticipate the complexities of coordinated effort, dearth of information about its design and the absence of a normative perspective to share responsibility. This paper addresses this ill-conception of policy integration by reconceptualising it through a qualitative examination of 16 road safety stakeholders’ written submissions, lodged with the Australian Transport Council in 2011. The resulting, new principle of policy integration, Participatory Deliberative Integration, provides a conceptual framework for the alignment of effort across stakeholders in transport, health, traffic law enforcement, relevant trades and the community. With the adoption of Participatory Deliberative Integration, road safety management agencies should secure the commitment of key stakeholders in the development and implementation of, amongst other policy measures, National Road Safety Strategies and Mix Mode Integrated Timetabling.

The changing role of actors in the participatory project for rural craftspeople

This paper examines the role of actors in a participatory project, a case study of the glass-bead traditional craft industry in rural Indonesia. The project aimed to unite and empower rural craftspeople with regard to their unique potential. The problems of empowering rural craftspeople were complicated, due to the interrelated aspect of rural community life, cultural and educational backgrounds, as well as the local political situation. However, through a comprehensive understanding of the community prior to the project and by maintaining the communication, craftspeople were engaged actively in the project by promoting the craft industry to local buyers. The researcher, other facilitators and the community leader gave supportive roles at the middle and the end stage of the project.

Participatory approach to support community development of rural craftspeople

This paper presents a case study of the participatory project in the Jombang glass bead craft industry. Economic instability has brought significant business challenges in the community. The involvement of outsiders to collaborate with craftspeople in order to support business innovation as well as strengthen the social capital in the community is essential. However, facilitating a rural community to formulate and implement bottom-up planning needs an integrated approach. In this paper, we explain a participatory project in the rural craftspeople community that resulted in a collective action. The project aimed at uniting and empowering rural craftspeople focusing on the unique skills and knowledge of participants. There are some aspects influencing the success of collective action: the ability to understand the local political situation; the role of facilitators to respect and support the unique potential of craftspeople; and the economic benefit of the program.

Re-framing infrastructure investment decision-making processes : a preliminary scoping study for urban flood risk management in Jakarta, Indonesia

Jakarta, Indonesia’s chronic housing shortage poses multiple challenges for contemporary policy-makers. While it may be in the city’s interest to increase the availability of housing, there is limited land to do so. Market pressures, in tandem with government’s desire for housing availability, demand consideration of even marginal lands, such as those within floodplains, for development. Increasingly, planning for a flood resilient Jakarta is complicated by a number of factors, including: the city is highly urbanized and land use data is limited; flood management is technically complex, creating potential barriers to engagement for both decision-makers and the public; inherent uncertainty exists throughout modelling efforts, central to management; and risk and liability for infrastructure investments is unclear. These obstacles require localized watershed-level participatory planning to address risks of flooding where possible and reduce the likelihood that informal settlements occur in areas of extreme risk. This paper presents a preliminary scoping study for determination of an effective participatory planning method to encourage more resilient development. First, the scoping study provides background relevant to the challenges faced in planning for contemporary Jakarta. Second, the study examines the current use of decision-support tools, such as Geographic Information Systems (GIS), in planning for Jakarta. Existing capacity in the use of GIS allows for consideration of the use of an emerging method of community consultation - Multi-Criteria Decision-Making (MCDM) support systems infused with geospatial information - to aid in engagement with the public and improve decision-making outcomes. While these methods have been used in Australia to promote stakeholder engagement in urban intensification, the planned research will be an early introduction of the method to Indonesia. As a consequence of this intervention, it is expected that planning activities will result in a more resilient city, capable of engaging with disaster risk management in a more effective manner.

Using participatory action research for heart self care amongst indigenous patients

This paper describes the initial phases of the Fluid Watchers Pacific Rim project: a participatory action research project that involves developing and trialling an iPad app to provide monitoring and self-care for Indigenous Australians with heart failure. The development phase involved working with health experts, an IT team and Indigenous heart-failure patients through three cycles of development and critical reflection. This was followed by a small pilot study to examine the app’s effectiveness. In this paper, the researchers explain why IT-supported health education can be successful in decreasing re-hospitalisation and improving self-management skills. They describe the steps they took to ensure community participation and ownership of the project and present the findings of their pilot study. This pilot project suggests that an iPad app may be a practical and successful way to provide health-care support for Indigenous Australian heart-failure patients.

It takes more than good intentions : institutional and attitudinal impediments to engaging young people in participatory planning

While participatory processes have become an important part of planning, young people are a particularly vulnerable group in terms of potential marginalisation and exclusion from effective participation. Including the views of young people in participatory planning is not simply a matter of bringing them into existing processes. Instead, participatory processes must find ways to integrate and accommodate their needs and ways of expressing their views. Without these adjustments young people may simply move from being kept outside the planning process to a situation where, although they are formally included, their claims are not taken seriously and they are not treated with equal respect. In this paper we reflect on the success of a community advisory committee, formed to consider water planning issues, in integrating the views of young people into their deliberations. Using Iris Marion Young’s (1995) ideas of communicative democracy we highlight the challenges and opportunities presented by this participatory approach, as articulated by both the young people involved and the adult participants. We specifically consider how the elements of greeting, rhetoric and narrative were reflected in the committee process. We argue that both planners and adult participants need to ensure that participatory processes allow for the equal engagement of all participants and place equal value on their contributions. Our research shows that this involves both an institutional and attitudinal commitment to include the views of young people. The institutional commitment requires young people to be included in processes and for their involvement to be supported. However, the attitudinal commitment it is equally important and requires that adult participants be prepared not only to accept the views of younger participants but to actively encourage and support their full participation.

Building citizens : participatory planning practice and a transformative politics of difference

Decision makers frequently use separate participatory activities to involve marginalised groups. This approach can generate valuable insights, but it has limitations. We discuss the benefits and limits through two examples involving young people, and outline how the approach can be modified, thereby building citizens who are responsive to other perspectives.

Engaging young people : reflections from a participatory water planning exercise

While participatory processes have become an important part of water planning, young people are a particularly vulnerable group in terms of potential marginalisation and exclusion from effective participation. Including the views of young people in water planning is not simply a matter of bringing them into existing processes. Instead, processes must be modified to accommodate their needs and ways of expressing their views. Without these adjustments young people may simply move from being kept outside the process to a situation where although they are formally included, their claims are not taken seriously and they are not treated with equal respect. In this paper we reflect on the success of the community advisory committee, formed to develop the Gold Coast Waterfuture Strategy, in integrating the views of young people into their deliberations. Using Young's communicative democracy we highlight the challenges and opportunities presented by this approach, as articulated by both the young people involved and the adult participants, and specifically consider the how the elements of greeting, rhetoric and narrative were reflected in the committee process.

Putting people in planning: participatory democracy, community building and social inclusion

Like other Western nations, since the nineteenth century Australia has been a representative democracy, in which citizens elect parliamentary representatives to make decisions and develop policies on their behalf (see chapter 5). These representatives are supported in their decision making by an ‘arm’s-length’, ‘techno-bureaucratic administration’, which includes experts such as environmental planners (Fung and Wright 2003, p. 3). However, as the issues for decision-making become increasingly complex, and societies increasingly diverse, the idea of citizen participation in decision-making is ever more accepted. There is now a significant body of political theory arguing for a more participatory model of democracy (participatory democracy), a model that strives to create opportunities for all members of a society to contribute meaningfully to decisions about the matters affecting their lives.

A participatory framework for researching and evaluating communication for development and social change

The growing dominance of project planning cycles and results-based management in development over the past 20 years has significant implications for the effective evaluation of communication for development and social change and the sustainability of these processes. These approaches to development and evaluation usually give priority to the linear, logical framework (or log frame) approach promoted by many development institutions. This tends to emphasize upward accountability approaches to development and its evaluation, so that development is driven by exogenous rather than endogenous models of development and social change. Such approaches are underpinned by ideas of preplanning, and predetermination of what successful out -comes look like. In this way, outcomes of complex interventions tend to be reduced to simple, cause-effect processes and the categorization of things, including people (Chambers and Pettit 2004; Eyben 2011). This runs counter to communication for development approaches, which prioritize engagement, relationships, empowerment and dialogue as important components for positive social change.

The design and development of music eScape: A new iPhone mood management music App

The physical, emotional, educational and social developmental challenges of adolescence can be associated with high levels of emotional vulnerability. Thus, the development of effective emotion-regulation strategies is crucial during this time period. Young people commonly use music to identify, express and regulate their emotions. Modern mobile technology provides an engaging, easily accessible means of assisting young people through music. A systematic contextual review identified 20 iPhone applications addressing emotions through music and two independent raters, using the Mobile App Rating Scale (MARS), evaluated the quality of the apps. Their characteristics, key features and overall quality will be presented. Three participatory design workshops (N=13, 6 males, 7 females; age 15-25) were conducted to explore young people’s use of music to enhance wellbeing. Young people were also asked to trial existing mood and music apps and to conceptualise their ultimate mood targeting music application. A thematic analysis of the participatory design workshops content identified the following music affect-regulation strategies: relationship building, modifying cognitions, modifying emotions, and immersing in emotions. The application of the key learnings from the mobile app review and participatory design workshops and the design and development of the music eScape app were presented and implications for future research was discussed.

Little boxes : changing perceptions and including young people in socially sustainable participatory planning

In recent years, a great deal has been written about the benefits and ethics of including young people in participative decision-making. This has been accompanied by a burgeoning interest in including their views in participatory planning exercises that has not always been realised in practice. Drawing on a detailed analysis of the perceptions of adults and young people involved in a participatory planning exercise on Australia‟s Gold Coast, we believe that there are two major hurdles to the „full‟ engagement of young people that are in some respects two sides of the same coin: the sometimes paternalistic perceptions and often dismissive attitude that many adults have towards the participation of young people; and the perceptions that young people may have of themselves and their subordinate place in an adult-dominated planning environment. Together, such views act to place limitations on the participation of young people because they set up unrealistic expectations for both adult and younger participants in terms of how and why young people participate, and what this participation should „look and feel‟ like. In this paper, through the metaphor of boxes, we propose a number of issues that should be addressed when involving young people in participatory planning processes to ensure the most from their participation for all involved.

Smart cities: An effective urban development and management model?

It is argued that the smart cities model promise solutions to fuel sustainable development and a high quality of life with a wise management of natural resources, through participatory action and engagement. The paper provides a critical review of this model and application attempts of smart urban technologies in contemporary cities by particularly looking into emerging practices of ubiquitous eco-cities as exemplar smart cities initiatives. Through a thorough review of literature and best practices on the smart cities model, this paper attempts to address the research question of whether smart cities model is just another fashionable city brand or an effective urban development and management model to solve the problems of our cities. The findings shed light on urban planning and development considerations for the integration of smart urban technologies and their possible implications in shaping up of the built environment to produce prosperous and sustainable urban futures.

Democratic and participatory citizenship: Youth action for sustainability in Australia

Purpose – The purpose of this paper is to provide a critical analysis of recent examples of action competence among young people engaged in democratic participatory action in sustainability programs in Australia. It explores examples of priorities identified for citizen action, the forms this action takes and the ways that democratic participation can achieve positive outcomes for future sustainability. It suggests multiple ways for developing action competence that provides further opportunities for authentic and engaging citizen action for youth connected to school- and community-based learning, in new and powerful ways. Design/methodology/approach – This conceptual paper examines international literature on the theory of “action competence,” its significance for education for sustainability (EfS) and the ways it can inform education for young people’s democratic participatory citizenship and civic engagement. It analyses examples of the development of action competency among young people in Australia, including the problems and priorities identified for citizen action, the forms this action takes and how it can achieve positive outcomes for sustainability. Following this analysis, the paper suggests multiple ways for developing action competence in EfS in schools and communities in new and powerful ways. Findings – Developing EfS to increase democratic and participatory action among young citizens is now widely regarded as an urgent education priority. There are growing exemplars of school and community organizations’ involvement in developing EfS learning and teaching to increase participatory citizenship. Young people are being empowered to develop a greater sense of agency through involvement in programs that develop action competence with a focus on sustainability in and out of school. New forms of participation include student action teams and peer collaboration among youth who are marshaling social media and direction action to achieve change. Originality/value – It contributes to the literature on multiple ways for developing action competence in EfS.