Developing a participatory impact assessment approach and action research culture within a communication for social change organisation in Nepal

Developing an effective impact evaluation framework, managing and conducting rigorous impact evaluations, and developing a strong research and evaluation culture within development communication organisations presents many challenges. This is especially so when both the community and organisational context is continually changing and the outcomes of programs are complex and difficult to clearly identify.----- This paper presents a case study from a research project being conducted from 2007-2010 that aims to address these challenges and issues, entitled Assessing Communication for Social Change: A New Agenda in Impact Assessment. Building on previous development communication projects which used ethnographic action research, this project is developing, trailing and rigorously evaluating a participatory impact assessment methodology for assessing the social change impacts of community radio programs in Nepal. This project is a collaboration between Equal Access – Nepal (EAN), Equal Access – International, local stakeholders and listeners, a network of trained community researchers, and a research team from two Australian universities. A key element of the project is the establishment of an organisational culture within EAN that values and supports the impact assessment process being developed, which is based on continuous action learning and improvement. The paper describes the situation related to monitoring and evaluation (M&E) and impact assessment before the project began, in which EAN was often reliant on time-bound studies and ‘success stories’ derived from listener letters and feedback. We then outline the various strategies used in an effort to develop stronger and more effective impact assessment and M&E systems, and the gradual changes that have occurred to date. These changes include a greater understanding of the value of adopting a participatory, holistic, evidence-based approach to impact assessment. We also critically review the many challenges experienced in this process, including:----- • Tension between the pressure from donors to ‘prove’ impacts and the adoption of a bottom-up, participatory approach based on ‘improving’ programs in ways that meet community needs and aspirations.----- • Resistance from the content teams to changing their existing M&E practices and to the perceived complexity of the approach.----- • Lack of meaningful connection between the M&E and content teams.----- • Human resource problems and lack of capacity in analysing qualitative data and reporting results.----- • The contextual challenges, including extreme poverty, wide cultural and linguistic diversity, poor transport and communications infrastructure, and political instability.----- • A general lack of acceptance of the importance of evaluation within Nepal due to accepting everything as fate or ‘natural’ rather than requiring investigation into a problem.

The entrepreneurial vlogger : participatory culture beyond the professional-amateur divide

In this chapter, we frame YouTube as an example of “co-creative” culture – whatever YouTube is, it is produced dynamically (that is, as an ongoing process, over time) as a result of many interconnected instances of participation, by many different people. In order to understand these co-creative relationships, it is important not to focus exclusively on how the “ordinary consumer” or “amateur producer,” are participating in YouTube; rather, we argue it is necessary to include the activities of “traditional media” companies and media professionals, and more importantly, the new models of media entrepreneurialism that are grounded in YouTube’s “grassroots” culture. Hence, this chapter focuses the role that “YouTube stars” – highly visible and successful “homegrown” performers and producers – play in modelling and negotiating these co-creative relationships within the context of YouTube’s social network; and the new models of entrepreneurship within participatory culture that they represent.

Cipher cities

Cipher Cities was a practice-led research project developed in 3 stages between 2005 and 2007 resulting in the creation of a unique online community, ‘Cipher Cities’, that provides simple authoring tools and processes for individuals and groups to create their own mobile events and event journals, build community profile and participate in other online community activities. Cipher Cities was created to revitalise peoples relationship to everyday places by giving them the opportunity and motivation to create and share complex digital stories in simple and engaging ways. To do so we developed new design processes and methods for both the research team and the end user to appropriate web and mobile technologies. To do so we collaborated with ethnographers, designers and ICT researchers and developers. In teams we ran a series of workshops in a wide variety of cities in Australia to refine an engagement process and to test a series of iteratively developed prototypes to refine the systems that supported community motivation and collaboration. The result of the research is 2 fold: 1. a sophisticated prototype for researchers and designers to further experiment with community engagement methodologies using existing and emerging communications technologies. 2. A ‘human dimensions matrix’. This matrix assists in the identification and modification of place based interventions in the social, technical, spatial, cultural, pedagogical conditions of any given community. This matrix has now become an essential part of a number of subsequent projects and assists design collaborators to successfully conceptualise, generate and evaluate interactive experiences. the research team employed practice-led action research methodologies that involved a collaborative effort across the fields of interaction design and social science, in particular ethnography, in order to: 1. seek, contest, refine a design methodology that would maximise the successful application of a dynamic system to create new kinds of interactions between people, places and artefacts’. 2. To design and deploy an application that intervenes in place-based and mobile technologies and offers people simple interfaces to create and share digital stories. Cipher Cities was awarded 3 separate CRC competitive grants (over $270,000 in total) to assist 3 stages of research covering the development of the Ethnographic Design Methodologies, the development of the tools, and the testing and refinement of both the engagement models and technologies. The resulting methodologies and tools are in the process of being commercialised by the Australasian CRC for Interaction Design.

Ourplace : the convergence of locative media and online participatory culture

The trans-locative potential of the Internet has driven the design of many online applications. Online communities largely cluster around topics of interest, which take precedence over participants’ geographical locations. The site of production is often disregarded when creative content appears online. However, for some, a sense of place is a defining aspect of creativity. Yet environments that focus on the display and sharing of regionally situated content have, so far, been largely overlooked. Recent developments in geo-technologies have precipitated the emergence of a new field of interactive media. Entitled locative media, it emphasizes the geographical context of media. This paper argues that we might combine practices of locative media (experiential mapping and geo-spatial annotation) with aspects of online participatory culture (uploading, file-sharing and search categorization) to produce online applications that support geographically ‘located’ communities. It discusses the design considerations and possibilities of this convergence,making reference to an example, OurPlace 3G to 3D, which has to date been developed as a prototype.1 It goes on to discuss the benefits and potential uses of such convergent applications, including the co-production of spatial- emporal narratives of place.

User-generated content and the future of public broadcasting : a case study of the special broadcasting service

This thesis presents a case study of the Special Broadcasting Service documenting the broadcasting challenges posed by user-generated content initiatives and the work-place approach to strategies for participation. Using the action research method, the project findings reveal that limitations to resources and funding determined the scope for innovation and that the practice of executive editorial control over content was considered fundamental to fulfilling the responsibilities of the public service mandate. Media workers were overwhelmingly positive about the enhanced productive capabilities of the audience and willing to facilitate moderated interactions, however the effectiveness of these initiatives differed according to the level of skills required. This thesis demonstrates how participatory initiatives can enhance aspects of the public service remit relating to cultural diversity, the servicing of niche interests, and broader social representation, and help reinvigorate the relevance of public service broadcasting in the digitalised media sphere.

Transferring knowledge through peer-reviewed assessment : the creation of a community of practice and the threats to its survival

While it is generally accepted in the learning and teaching literature that assessment is the single biggest influence on how students approach their learning, assessment methods within higher education are generally conservative and inflexible. Constrained by policy and accreditation requirements and the need for the explicit articulation of assessment standards for public accountability purposes, assessment tasks can fail to engage students or reflect the tasks students will face in the world of practice. Innovative assessment design can simultaneously deliver program objectives and active learning through a knowledge transfer process which increases student participation. This social constructivist view highlights that acquiring an understanding of assessment processes, criteria and standards needs active student participation. Within this context, a peer-assessed, weekly, assessment task was introduced in the first “serious” accounting subject offered as part of an undergraduate degree. The positive outcomes of this assessment innovation was that student failure rates declined 15%, tutorial participation increased fourfold, tutorial engagement increased six-fold and there was a 100% approval rating for the retention of the assessment task. In contributing to the core conference theme of “seismic” shifts within higher education, in stark contrast to the positive student response, staff-related issues of assessment conservatism and the necessity of meeting increasing research commitments, threatened the assessment task’s survival. These opposing forces to change have the potential to weaken the ability of higher education assessment arrangements to adequately serve either a new generation of students or the sector's community stakeholders.

Transferring knowledge through peer-reviewed assessment : the creation of a community of practice and the threats to its survival

While it is generally accepted in the learning and teaching literature that assessment is the single biggest influence on how students approach their learning, assessment methods within higher education are generally conservative and inflexible. Constrained by policy and accreditation requirements and the need for the explicit articulation of assessment standards for public accountability purposes, assessment tasks can fail to engage students or reflect the tasks students will face in the world of practice. Innovative assessment design can simultaneously deliver program objectives and active learning through a knowledge transfer process which increases student participation. This social constructivist view highlights that acquiring an understanding of assessment processes, criteria and standards needs active student participation. Within this context, a peer-assessed, weekly, assessment task was introduced in the first “serious” accounting subject offered as part of an undergraduate degree. The positive outcomes of this assessment innovation was that student failure rates declined 15%, tutorial participation increased fourfold, tutorial engagement increased six-fold and there was a 100% approval rating for the retention of the assessment task. In contributing to the core conference theme of “seismic” shifts within higher education, in stark contrast to the positive student response, staff-related issues of assessment conservatism and the necessity of meeting increasing research commitments, threatened the assessment task’s survival. These opposing forces to change have the potential to weaken the ability of higher education assessment arrangements to adequately serve either a new generation of students or the sector's community stakeholders.

Mobilizing the brand experience: The adoption and use of mobile media in building consumer relationships in China

This research examines why and how brand owners in China adopt and use mobile media in marketing campaigns to deliver co-creation brand experiences and build consumer relationships. China represents an interesting case to study as it has leapfrogged into the age of consumer society and mobile media adoption. As the largest mobile market globally, it has experienced the intensity of mobile technology diffusion; and with it the rise of mobile consumer culture and participatory culture. Further, the rising individualism and the socio-cultural heritage in collectivism serve as a structuring tension in how mobile media is leveraged in marketing to cater to consumers' desires for individuality and social interaction. First, through expert interviews guided by the technology-organization-environment (TOE) framework (Tornatzky & Fleischer, 1990) as well as integrating innovation diffusion theory (E. Rogers, 2003), this research attempts to fill the gap of theoretical application in mobile marketing adoption at the firm level in China, and unravel the adoption factors of mobile marketing by brand owners in China. In total, 27 semi-structured interviews were conducted with key industry informants from mobile agencies, traditional agencies, venture capital firms, mobile content and service providers, mobile portals, and marketing management at brand owners. Second, based on case studies in China, this research investigates the use of mobile marketing to facilitate innovative co-creation of brand experience to cater to both individualistic as well as collective tendencies and desires amongst Chinese consumers. Through multiple case studies of the campaigns conducted by Nokia, Clean & Clear, and The North Face, and informed by in-depth interviews and document analysis, this research analyses the role of mobile media in marketing campaigns along three dimensions: the role of mobile media in content generation and consumption, the centrality of mobile media as text, tools or platforms; and the interactive environment. Specifically, the cases are organized along the spectrum from user-generated content to corporate-generated content, mobile media's role from being supplementary to it being central, and from a virtual environment to a hybrid environment. Overall, these cases demonstrate how brand owners adapt mobile media as text, tools, platforms, and environments to deliver co-creation brand experiences exploiting both individualistic as well as collective tendencies and desires amongst Chinese consumers. This research contributes to the literature on firm adoption of mobile marketing, and the role of the mobile media in facilitating co-creation experiences for Chinese consumers. It develops a model of the technological, organizational and environmental factors influencing mobile marketing adoption by firms, and provides a model explaining the role of mobile media in facilitating brand experience co-creation. The findings also demonstrate that mobile media can be leveraged to facilitate co-creation brand experience to generate added value; and meanwhile cater to both the rising individualism and the deep-seated collectivism of Chinese consumers. Empirically, it assists industry practitioners in understanding the adoption of mobile marketing in China, especially those on the supply side in order to improve their offerings and propositions. It also assists brand owners and agencies in designing their mobile marketing strategies to build consumer relationships in China.

Teen content creators : experiences of using information to learn

As access to networked digital communities increases, a growing number of teens participate in digital communities by creating and sharing a variety of content. The affordances of social media - ease of use, ubiquitous access, and communal nature - have made creating and sharing content an appealing process for teens. Teens primarily learn the practices of encountering and using information through social interaction and participation within digital communities. This article adopts the position that information literacy is the experience of using information to learn. It reports on an investigation into teens experiences in the United States, as they use information to learn how to create content and participate within the context of social media. Teens that participate in sharing art on sites such as DeiviantArt, website creation, blogging, and/or posting edited videos via YouTube and Vimeo, were interviewed. The interviews explored teens' information experiences within particular social and digital contexts. Teens discussed the information they used, how information was gathered and accessed, and explored the process of using that information to participate in the communities.

"Participation" in participatory design research

This thesis is about defining participation in the context of fostering research cohesion in the field of Participatory Design. The systematic and incremental building of new knowledge is the process by which science and research is advanced. This process requires a certain type of cohesion in the way research is undertaken for new knowledge to be built from the knowledge provided by previous projects and research. To support this process and to foster research cohesion three conditions are necessary. These conditions are: common ground between practitioners, problem-space positioning, and adherence to clear research criteria. The challenge of fostering research cohesion in Participatory Design is apparent in at least four themes raised in the literature: the role of politics within Participatory Design epistemology, the role of participation, design with users, and the ability to translate theory into practice. These four thematic challenges frame the context which the research gap is situated. These themes are also further investigated and the research gap – a general lack of research cohesion – along with one avenue for addressing this gap – a clear and operationalizable definition for participation – are identified. The intended contribution of this thesis is to develop a framework and visual tool to address this research gap. In particular, an initial approximation for a clear and operationalizable definition for participation will be proposed such that it can be used within the field of Participatory Design to run projects and foster research cohesion. In pursuit of this contribution, a critical lens is developed and used to analyse some of the principles and practices of Participatory Design that are regarded as foundational. This lens addresses how to define participation in a way that adheres to basic principles of scientific rigour – namely, ensuring that the elements of a theory are operationalizable, falsifiable, generalizable, and useful, and it also treats participation as a construct rather than treating the notion of participation as a variable. A systematic analysis is performed using this lens on the principles and practices that are considered foundational within the field. From this analysis, three components of the participation construct – impact, influence, and agency – are identified. These components are then broken down into two constituent variables each (six in all) and represented visually. Impact is described as the relationship between the quality and use of information. Influence is described as the relationship between the amount and scope of decision making. Agency is described as the relationship between the motivation of the participant and the solidarity of the group. Thus, as a construct, participation is described as the relationship between a participant’s impact, influence, and agency. In the concluding section, the value of this participation construct is explored for its utility in enhancing project work and fostering research cohesion. Three items of potential value that emerge are: the creation of a visual tool through the representation of these six constituent variables in one image; the elaboration of a common language for researchers based on the six constituent variables identified; and the ability to systematically identify and remedy participation gaps throughout the life of the project. While future research exploring the applicability of the participation construct in real world projects is necessary, it is intended that this initial approximation of a participation construct in the form of the visual tool will serve as the basis for a cohesive and rigorous discussion about participation in Participatory Design.

Design process excludes users : the co-creation activities between user and designer

User needs are a fundamental element of design. If the design process does not properly reflect user needs, the design will be severely compromised. Therefore, it is worthwhile to investigate how the user is, and user needs are, understood in the design process. In this article, three accepted linear process models for web site and interactive media design are reviewed in terms of the designer and user participation. The article then proposes a user-evolving collaborative design process which is built on co-creation activities between designer and user. Co-creation activities across the entire design process structurally and ontologically reposition the users, and user needs, centrally, which allows the designers to holistically approach to the user needs through building a partnership with the users. Co-creation creates an equal evolving participatory process between user and designer towards sharing values and knowledge and creating new domains of collective creativity.

Meta-evaluation to improve learning, evaluation capacity development and sustainability : Findings from a participatory evaluation project in Nepal

Recent studies suggest that meta-evaluation can be valuable in developing new approaches to evaluation, building evaluation capacities, and enhancing organizational learning. These new extensions of the concept of meta-evaluation are significant, given the growing emphasis on improving the quality and effectiveness of evaluation practices in the South Asian region. Following a review of the literature, this paper presents a case study of the use of concurrent meta-evaluation in the four-year project Assessing Communication for Social Change which developed and trialled a participatory impact assessment methodology in collaboration with a development communication Non-government organization (NGO) in Nepal. Key objectives of the meta-evaluation included to: continuously develop, adapt and improve the impact assessment methodology, Monitoring and Evaluation (M&E) systems and process and other project activities; identify impacts of the project; and build capacities in critical reflection and review. Our analysis indicates that this meta-evaluation was essential to understanding various constraints related to the organizational context that affected the success of the project and the development of improved M&E systems and capacities within the NGO. We identified several limitations of our meta-evaluation methods, which were balanced by the strengths of other methods. Our case study suggests that as well as assessing the quality, credibility and value of evaluation practices, meta-evaluations need to focus on important contextual issues that can have significant impacts on the outcomes of participatory evaluation projects. They include hierarchical organizational cultures, communication barriers, power/knowledge relations, and the time and resources available. Meta-evaluations also need to consider wider issues such as the sustainability of evaluation systems and approaches.

Reconceptualising mega project success in Australian Defence : recognising the importance of value co-creation

Recent literature in project management has urged a re-conceptualisation of projects as a value co-creation process. Contrary to the traditional output-focused project methodology, the value creation perspective argues for the importance of creating new knowledge, processes, and systems for suppliers and customers. Stakeholder involvement is important in this new perspective, as the balancing of competing needs of stakeholders in mega projects becomes a major challenge in managing the value co-creation process. In this study we present interview data from three Australian defence mega projects to demonstrate that senior executives have a more complex understanding of project success than traditional iron triangle measures. In these mega defence projects, customers and other stakeholders actively engage in the value creation process, and over time both content and process value are created to increase defence and national capability. Value created and captured during and post projects are the key to true success.

Information practices of teen content creators : the intersection of action and experiences a grounded theory study

What are the information practices of teen content creators? In the United States over two thirds of teens have participated in creating and sharing content in online communities that are developed for the purpose of allowing users to be producers of content. This study investigates how teens participating in digital participatory communities find and use information as well as how they experience the information. From this investigation emerged a model of their information practices while creating and sharing content such as film-making, visual art work, story telling, music, programming, and web site design in digital participatory communities. The research uses grounded theory methodology in a social constructionist framework to investigate the research problem: what are the information practices of teen content creators? Data was gathered through semi-structured interviews and observation of teen’s digital communities. Analysis occurred concurrently with data collection, and the principle of constant comparison was applied in analysis. As findings were constructed from the data, additional data was collected until a substantive theory was constructed and no new information emerged from data collection. The theory that was constructed from the data describes five information practices of teen content creators. The five information practices are learning community, negotiating aesthetic, negotiating control, negotiating capacity, and representing knowledge. In describing the five information practices there are three necessary descriptive components, the community of practice, the experiences of information and the information actions. The experiences of information include information as participation, inspiration, collaboration, process, and artifact. Information actions include activities that occur in the categories of gathering, thinking and creating. The experiences of information and information actions intersect in the information practices, which are situated within the specific community of practice, such as a digital participatory community. Finally, the information practices interact and build upon one another and this is represented in a graphic model and explanation.