Meta-evaluation to improve learning, evaluation capacity development and sustainability : Findings from a participatory evaluation project in Nepal

Recent studies suggest that meta-evaluation can be valuable in developing new approaches to evaluation, building evaluation capacities, and enhancing organizational learning. These new extensions of the concept of meta-evaluation are significant, given the growing emphasis on improving the quality and effectiveness of evaluation practices in the South Asian region. Following a review of the literature, this paper presents a case study of the use of concurrent meta-evaluation in the four-year project Assessing Communication for Social Change which developed and trialled a participatory impact assessment methodology in collaboration with a development communication Non-government organization (NGO) in Nepal. Key objectives of the meta-evaluation included to: continuously develop, adapt and improve the impact assessment methodology, Monitoring and Evaluation (M&E) systems and process and other project activities; identify impacts of the project; and build capacities in critical reflection and review. Our analysis indicates that this meta-evaluation was essential to understanding various constraints related to the organizational context that affected the success of the project and the development of improved M&E systems and capacities within the NGO. We identified several limitations of our meta-evaluation methods, which were balanced by the strengths of other methods. Our case study suggests that as well as assessing the quality, credibility and value of evaluation practices, meta-evaluations need to focus on important contextual issues that can have significant impacts on the outcomes of participatory evaluation projects. They include hierarchical organizational cultures, communication barriers, power/knowledge relations, and the time and resources available. Meta-evaluations also need to consider wider issues such as the sustainability of evaluation systems and approaches.

Information practices of teen content creators : the intersection of action and experiences a grounded theory study

What are the information practices of teen content creators? In the United States over two thirds of teens have participated in creating and sharing content in online communities that are developed for the purpose of allowing users to be producers of content. This study investigates how teens participating in digital participatory communities find and use information as well as how they experience the information. From this investigation emerged a model of their information practices while creating and sharing content such as film-making, visual art work, story telling, music, programming, and web site design in digital participatory communities. The research uses grounded theory methodology in a social constructionist framework to investigate the research problem: what are the information practices of teen content creators? Data was gathered through semi-structured interviews and observation of teen’s digital communities. Analysis occurred concurrently with data collection, and the principle of constant comparison was applied in analysis. As findings were constructed from the data, additional data was collected until a substantive theory was constructed and no new information emerged from data collection. The theory that was constructed from the data describes five information practices of teen content creators. The five information practices are learning community, negotiating aesthetic, negotiating control, negotiating capacity, and representing knowledge. In describing the five information practices there are three necessary descriptive components, the community of practice, the experiences of information and the information actions. The experiences of information include information as participation, inspiration, collaboration, process, and artifact. Information actions include activities that occur in the categories of gathering, thinking and creating. The experiences of information and information actions intersect in the information practices, which are situated within the specific community of practice, such as a digital participatory community. Finally, the information practices interact and build upon one another and this is represented in a graphic model and explanation.

Tech-Savvy Youth and Participatory Research in “iScapes”

This chapter provides a theoretically and empirically grounded discussion of participatory research methodologies with respect to investigating the dynamic and evolving phenomenon of young people constructing identities and social relations in “iScapes”. We coin the term iScapes to refer to the online/offline interconnectedness of spaces in the fabric of everyday life. Specifically, we offer a critical analysis of the participatory research methods we used in our own research project to understand the ways in which high school students use new media and information communication technologies (ICTs) to construct identities, form social relations, and engage in creative practices as part of their everyday lives. The chapter concludes with reflexive deliberations on our approach to participatory research that may benefit other researchers who share a similar interest in youth and new media.

Outdoor worker sun protection project : a comprehensive applied research project to demonstrate the effectiveness of sun protection measures which influence high risk outdoor workers in Queensland to adopt sun safe behaviour practices 2010 – 2013

The QUT Outdoor Worker Sun Protection (OWSP) project undertook a comprehensive applied health promotion project to demonstrate the effectiveness of sun protection measures which influence high risk outdoor workers in Queensland to adopt sun safe behaviours. The three year project (2010-2013) was driven by two key concepts: 1) The hierarchy of control, which is used to address risks in the workplace, advocates for six control measures that need to be considered in order of priority (refer to Section 3.4.2); and 2) the Ottawa Charter which recommends five action means to achieve health promotion (refer to Section 2.1). The project framework was underpinned by a participatory action research approach that valued peoples’ input, took advantage of existing skills and resources, and stimulated innovation (refer to Section 4.2). Fourteen workplaces (small and large) with a majority outdoor workforce were recruited across regional Queensland (Darling Downs, Northwest, Mackay and Cairns) from four industries types: 1) building and construction, 2) rural and farming, 3) local government, and 4) public sector. A workplace champion was identified at each workplace and was supported (through resource provision, regular contact and site visits) over a 14 to 18 month intervention period to make sun safety a priority in their workplace. Employees and employers were independently assessed for pre- and postintervention sun protection behaviours. As part of the intervention, an individualised sun safety action plan was developed in conjunction with each workplace to guide changes across six key strategy areas including: 1) Policy (e.g., adopt sun safety practices during all company events); 2) Structural and environmental (e.g., shade on worksites; eliminate or minimise reflective surfaces); 3) Personal protective equipment (PPE) (e.g., trial different types of sunscreens, or wide-brimmed hats); 4) Education and awareness (e.g., include sun safety in inductions and toolbox talks; send reminder emails or text messages to workers);5) Role modelling (e.g., by managers, supervisors, workplace champions and mentors); and 6) Skin examinations (e.g., allow time off work for skin checks). The participatory action process revealed that there was no “one size fits all” approach to sun safety in the workplace; a comprehensive, tailored approach was fundamental. This included providing workplaces with information, resources, skills, know how, incentives and practical help. For example, workplaces engaged in farming complete differing seasonal tasks across the year and needed to prepare for optimal sun safety of their workers during less labour intensive times. In some construction workplaces, long pants were considered a trip hazard and could not be used as part of a PPE strategy. Culture change was difficult to achieve and workplace champions needed guidance on the steps to facilitate this (e.g., influencing leaders through peer support, mentoring and role modelling). With the assistance of the project team the majority of workplaces were able to successfully implement the sun safety strategies contained within their action plans, up skilling them in the evidence for sun safety, how to overcome barriers, how to negotiate with all relevant parties and assess success. The most important enablers to the implementation of a successful action plan were a pro-active workplace champion, strong employee engagement, supportive management, the use of highly visual educational resources, and external support (provided by the project team through regular contact either directly through phone calls or indirectly through emails and e-newsletters). Identified barriers included a lack of time, the multiple roles of workplace champions, (especially among smaller workplaces), competing issues leading to a lack of priority for sun safety, the culture of outdoor workers, and costs or budgeting constraints. The level of sun safety awareness, knowledge, and sun protective behaviours reported by the workers increased between pre-and post-intervention. Of the nine sun protective behaviours that were assessed, the largest changes reported included a 26% increase in workers who “usually or always” wore a broad-brimmed hat, a 20% increase in the use of natural shade, a 19% increase in workers wearing long-sleeved collared shirts, and a 16% increase in workers wearing long trousers.

The role of the cultural intermediary within professional participatory communication : lessons from ABC Pool

As media institutions are encouraged to explore new production methodologies in the current economic crisis, they align with Schumpeter’s creative destruction provocation by exhibiting user-led political, organisation and socio-technical innovations. This paper highlights the significance of the cultural intermediary within the innovative, co-creative production arrangements for cultural artefacts by media professionals in institutional online communities. An institutional online community is defined as one that is housed, resourced and governed by commercial or non- commercial institutions and is not independently facilitated. Web 2.0 technologies have mobilised collaborative peer production activities for online content creation and professional media institutions face challenges in engaging participatory audiences in practices that are beneficial for all concerned stakeholders. The interests of those stakeholders often do not align, highlighting the need for an intermediary role that understands and translates the norms, rhetoric tropes and day-to-day activities between the individuals engaging in participatory communication activities for successful negotiation within the production process. This paper specifically explores the participatory relationship between the public service broadcaster (PSB), the Australian Broadcasting Corporation (ABC) and one of its online communities, ABC Pool (www.abc.net.au/pool). ABC Pool is an online platform developed and resourced by the ABC to encourage co-creation between audience members engaging in the production of user-generated content (UGC) and the professional producers housed within the ABC Radio Division. This empirical research emerges from a three-year research project where I employed an ethnographic action research methodology and was embedded at the ABC as the community manager of ABC Pool. In participatory communication environments, users favour meritocratic heterarchical governance over traditional institutional hierarchical systems (Malaby 2009). A reputation environment based on meritocracy requires an intermediary to identify the stakeholders, understand their interests and communicate effectively between them to negotiate successful production outcomes (Bruns 2008; Banks 2009). The community manager generally occupies this role, however it has emerged that other institutional production environments also employ an intermediary role under alternative monikers(Hutchinson 2012). A useful umbrella term to encompass the myriad of roles within this space is the cultural intermediary. The ABC has experimented with three institutional online community governance models that engage in cultural intermediation in differing decentralised capacities. The first and most closed is a single point of contact model where one cultural intermediary controls all of the communication of the participatory project. The second is a model of multiple cultural intermediaries engaging in communication between the institutional online community stakeholders simultaneously. The third is most open yet problematic as it promotes and empowers community participants to the level of cultural intermediaries. This paper uses the ABC Pool case study to highlight the differing levels of openness within cultural intermediation during the co-creative production process of a cultural artifact.

Making media participatory: Digital storytelling

From journalism to social media, the practices of our professional storytellers continue to evolve and change along with our storytelling institutions and their functions. Comprehending these developments is a key problem of contemporary media and cultural studies. Are the politics of representation giving way to a new progressive politics of self-representation and direct participation? Or, instead, are these new genres of self-representation part of a more general demotic turn in the function of contemporary media? Do media merely mediate or amplify cultural identities, or is media functionality becoming, closer to that of a translator or even an author of identities? How can we know if the changing actor-networks of storytelling contribute to a wider democratisation, a reshaping of the hierarchies of voice and agency? This chapter considers the place of one specific critical participatory media production practice known as 'digital storytelling’ in addressing these larger questions of socio-cultural change.

DIY media architecture: Open and participatory approaches to community engagement

Media architecture’s combination of the digital and the physical can trigger, enhance, and amplify urban experiences. In this paper, we examine how to bring about and foster more open and participatory approaches to engage communities through media architecture by identifying novel ways to put some of the creative process into the hands of laypeople. We review technical, spatial, and social aspects of DIY phenomena with a view to better understand maker cultures, communities, and practices. We synthesise our findings and ask if and how media architects as a community of practice can encourage the ‘open-sourcing’ of information and tools allowing laypeople to not only participate but become active instigators of change in their own right. We argue that enabling true DIY practices in media architecture may increase citizen control. Seeking design strategies that foster DIY approaches, we propose five areas for further work and investigation. The paper begs many questions indicating ample room for further research into DIY Media Architecture.

Working on the edge : Exploring the creative tensions experienced by practitioners facilitating participatory media projects with refugees and asylum seekers in Australia

This paper aims to provide a contextualised and embedded exploration of how the notions of "practice" and "participation", key concepts in the study of culture and media, are manifest in an example of a complex creative project. This project aimed to engage with refugees and asylum seekers through the co-creation of cultural material and is an outcome of an? ethnographic action research (Tacchi et al. 2003) partnership involving a community development worker in a settlement support agency and a storytelling/community media researcher (the author), along with other project collaborators. The discussion of this project focuses on the role of the facilitator and illustrates the processes of orchestrating a complex project involving a series of linked stages with cumulative effect. As practitioners at this site we are working in the space where personal narratives, participatory arts and media, and the staging of intercultural, civic dialogue events, intersect. Co-creative media facilitation in these contexts involves both managing hybrid communicative spaces and (re)combining the "integrative practices" (Schatzki 1996) of a range of professional approaches and creative roles. This is liminal work, located on the boundaries of several disciplines and practices. Drawing on reflections gathered from collaborative ethnographic descriptions (Bhattacharya 2008), this paper traces moments of practitioner uncertainty that can be linked to the way "practice" and “participation” is problematised within the community cultural development field in a way that is at times an uneasy fit with conventional ways of operating in social service roles. These moments of tension also indicate where this project pushed practitioners into spaces of improvisation and new learning. Keywords: Youth, refugees, community cultural development, co-creative media facilitation, ethnographic action research, intercultural dialogue.

Participatory action research for civic engagement

The future of civic engagement is characterised by both technological innovation as well as new technological user practices that are fuelled by trends towards mobile, personal devices; broadband connectivity; open data; urban interfaces; and, cloud computing. These technology trends are progressing at a rapid pace, and have led global technology vendors to package and sell the ‘Smart City’ as a centralized service delivery platform predicted to optimize and enhance cities’ key performance indicators – and generate a profitable market. The top-down deployment of these large and proprietary technology platforms have helped sectors such as energy, transport, and healthcare to increase efficiencies. However, an increasing number of scholars and commentators warn of another ‘IT bubble’ emerging. Along with some city leaders, they argue that the top-down approach does not fit the governance dynamics and values of a liberal democracy when applied across sectors. A thorough understanding is required, of the socio-cultural nuances of how people work, live, play across different environments, and how they employ social media and mobile devices to interact with, engage in, and constitute public realms. Although the term ‘slacktivism’ is sometimes used to denote a watered down version of civic engagement and activism that is reduced to clicking a ‘Like’ button and signing online petitions, we believe that we are far from witnessing another Biedermeier period that saw people focus on the domestic and the non-political. There is plenty of evidence to the contrary, such as post-election violence in Kenya in 2008, the Occupy movements in New York, Hong Kong and elsewhere, the Arab Spring, Stuttgart 21, Fukushima, the Taksim Gezi Park in Istanbul, and the Vinegar Movement in Brazil in 2013. These examples of civic action shape the dynamics of governments, and in turn, call for new processes to be incorporated into governance structures. Participatory research into these new processes across the triad of people, place and technology is a significant and timely investment to foster productive, sustainable, and livable human habitats. With this chapter, we want to reframe the current debates in academia and priorities in industry and government to allow citizens and civic actors to take their rightful centerpiece place in civic movements. This calls for new participatory approaches for co-inquiry and co-design. It is an evolving process with an explicit agenda to facilitate change, and we propose participatory action research (PAR) as an indispensable component in the journey to develop new governance infrastructures and practices for civic engagement. This chapter proposes participatory action research as a useful and fitting research paradigm to guide methodological considerations surrounding the study, design, development, and evaluation of civic technologies. We do not limit our definition of civic technologies to tools specifically designed to simply enhance government and governance, such as renewing your car registration online or casting your vote electronically on election day. Rather, we are interested in civic media and technologies that foster citizen engagement in the widest sense, and particularly the participatory design of such civic technologies that strive to involve citizens in political debate and action as well as question conventional approaches to political issues (DiSalvo, 2012; Dourish, 2010; Foth et al., 2013). Following an outline of some underlying principles and assumptions behind participatory action research, especially as it applies to cities, we will critically review case studies to illustrate the application of this approach with a view to engender robust, inclusive, and dynamic societies built on the principles of engaged liberal democracy. The rationale for this approach is an alternative to smart cities in a ‘perpetual tomorrow,’ (cf. e.g. Dourish & Bell, 2011), based on many weak and strong signals of civic actions revolving around technology seen today. It seeks to emphasize and direct attention to active citizenry over passive consumerism, human actors over human factors, culture over infrastructure, and prosperity over efficiency. First, we will have a look at some fundamental issues arising from applying simplistic smart city visions to the kind of a problem a city is (cf. Jacobs, 1961). We focus on the touch points between “the city” and its civic body, the citizens. In order to provide for meaningful civic engagement, the city must provide appropriate interfaces.

A comparative study of two design charrettes: Implications for codesign and participatory action research

The purpose of this paper is to explore how participatory prototyping, through the use of design charrettes, can advance participatory action research (PAR) approaches and contribute to codesign practices in organisational settings. This will be achieved through the comparison of two varying design charrette experiences from a PAR initiative to redesign spaces in the Auraria Library in Denver, Colorado. Each design charrette followed a three-stage sequence of information sharing, idea generation and prototyping, and prioritisation with each stage building upon the former, both in terms of design concepts and in building up elements of ‘making’. While both charrette structures were similar, leadership and execution varied considerably. Lessons learned from the two design charrette experiences are presented, including the value of participatory prototyping within PAR to support ‘research through design’ activities. In addition, it highlights the value of authentic design participation of ‘designing with’ rather than ‘designing for’ to encourage optimal design outcomes.

Voice, deliberation, resistance and persuasion through participatory journalism business models

The pervasive use of the World Wide Web by the general population has created a cultural shift throughout the world. It has enabled more people to share more information about more events and issues than was possible before its general use. As a consequence, it has transformed traditional news media’s approach to almost every aspect of journalism, with many organisations restructuring their philosophy and practice to include a variety of participatory spaces/forums where people are free to engage in deliberative dialogue about matters of public importance. This paper draws from an international collective case study that showcases various approaches to participatory online news journalism during the period 1997–2011 (Adams, 2013). The research finds differences in the ways in which public service, commercial, and independent news media give voice to the public, and ultimately in their approach to journalism’s role as the Fourth Estate––one of the key institutions of democracy. The work is framed by the notion that journalism in democratic societies has a crucial role in ensuring citizens are informed and engaged with public affairs. An examination of four media models, OhmyNews International, News Corp Australia (formerly News Limited), the Guardian and the British Broadcasting Corporation (BBC), showcases the various approaches to participatory online news journalism and how each provides different avenues for citizen engagement. Semistructured in-depth interviews with some of the key senior journalists and editors provide specific information on comparisons between the distinctive practices in each of their employer organisations.

Improving knowledge and practices in fresh produce chains

A successful supply chain must delivery the right product, value and satisfaction to the end customer, and profitability for its participants. Critical to getting the product right is the practices used to produce and maintain product quality through the supply chain from production to sale to the end customer. This paper describes the approach used by a R&D team to add value to supply chains through improving knowledge and practices. The desired outcome is better produce quality for consumers and more control and less wastage for chain participants. The team worked with specific supply chains to identify areas for improvement and to develop, test and implement improved practices. The knowledge gained was communicated to the industry to gain wider adoption of results. Three conditions were identified as critical for practice change - motivation, knowledge, and capacity for change. For improvement in practices to occur, a business must be motivated and have the knowledge and capacity to improve. Two case studies of working with Australian supply chains (mango and melons) are presented to illustrate our participatory methodology. A key activity is monitoring produce quality and handling practices and conditions to demonstrate to participants the points where quality deterioration occurs in the supply chain. This participatory approach is successful because working with supply chain participants generates knowledge and solutions to real problems. It enables the participants to observe the effect of handling practices and conditions on produce quality, gain knowledge and assess the benefits of improvements. Where existing knowledge is not present, research is conducted to fill the knowledge gaps. IV International Conference on Managing Quality in Chains - The Integrated View on Fruits and Vegetables Quality

Connecting the space between design and research: Explorations in participatory research supervision

In this article we offer a single case study using an action research method for gathering and analysing data offering insights valuable to both design and research supervision practice. We do not attempt to generalise from this single case, but offer it as an instance that can improve our understanding of research supervision practice. We question the conventional ‘dyadic’ models of research supervision and outline a more collaborative model, based on the signature pedagogy of architecture: the design studio. A novel approach to the supervision of creatively oriented post-graduate students is proposed, including new approaches to design methods and participatory supervision that draw on established design studio practices. This model collapses the distance between design and research activities. Our case study involving Research Masters student supervision in the discipline of Architecture, shows how ‘connected learning’ emerges from this approach. This type of learning builds strong elements of creativity and fun, which promote and enhance student engagement. The results of our action research suggests that students learn to research more easily in such an environment and supervisory practices are enhanced when we apply the techniques and characteristics of design studio pedagogy to the more conventional research pedagogies imported from the humanities. We believe that other creative disciplines can apply similar tactics to enrich both the creative practice of research and the supervision of HDR students.

Constellations: A participatory, online application for research collaboration in higher education interdisciplinary courses

The research establishes a model for online learning centering on the needs of integrative knowledge practices. Through the metaphor of Constellations, the practice-based research explores the complexities of working within interdisciplinary learning contexts and the potential of tools such as the Folksonomy learning platform for providing necessary conceptual support.

Building teachers' pedagogy practices in reasoning, to improve students' dispositions towards mathematics

Drawing on participatory action research, this study identifies the pedagogies necessary to advance reasoning, which is one of the proficiencies from the Australian Curriculum Mathematics, and explores how reasoning leads to greater productive disposition. With the current emphasis on STEM in schools, this research is timely. This thesis makes an original and substantive contribution to the understanding of why and how teachers can most effectively advance student proficiency in reasoning through targeted instructional strategies and style of instruction. The study explores the ways in which teacher practices, when focused on reasoning, enhance the disposition of students towards greater mathematical proficiency.