Increasing the rigour and trustworthiness of participatory evaluations : learnings from the field

Participatory evaluation and participatory action research (PAR) are increasingly used in community-based programs and initiatives and there is a growing acknowledgement of their value. These methodologies focus more on knowledge generated and constructed through lived experience than through social science (Vanderplaat 1995). The scientific ideal of objectivity is usually rejected in favour of a holistic approach that acknowledges and takes into account the diverse perspectives, values and interpretations of participants and evaluation professionals. However, evaluation rigour need not be lost in this approach. Increasing the rigour and trustworthiness of participatory evaluations and PAR increases the likelihood that results are seen as credible and are used to continually improve programs and policies.----- Drawing on learnings and critical reflections about the use of feminist and participatory forms of evaluation and PAR over a 10-year period, significant sources of rigour identified include:----- • participation and communication methods that develop relations of mutual trust and open communication----- • using multiple theories and methodologies, multiple sources of data, and multiple methods of data collection----- • ongoing meta-evaluation and critical reflection----- • critically assessing the intended and unintended impacts of evaluations, using relevant theoretical models----- • using rigorous data analysis and reporting processes----- • participant reviews of evaluation case studies, impact assessments and reports.

A participatory design agenda for ubiquitous computing : a case study in dental practice

This paper reflects upon our attempts to bring a participatory design approach to design research into interfaces that better support dental practice. The project brought together design researchers, general and specialist dental practitioners, the CEO of a dental software company and, to a limited extent, dental patients. We explored the potential for deployment of speech and gesture technologies in the challenging and authentic context of dental practices. The paper describes the various motivations behind the project, the negotiation of access and the development of the participant relationships as seen from the researchers' perspectives. Conducting participatory design sessions with busy professionals demands preparation, improvisation, and clarity of purpose. The paper describes how we identified what went well and when to shift tactics. The contribution of the paper is in its description of what we learned in bringing participatory design principles to a project that spanned technical research interests, commercial objectives and placing demands upon the time of skilled professionals.

Bridging technical and HCI research : creating usable ubiquitous computing

This paper describes methods used to support collaboration and communication between practitioners, designers and engineers when designing ubiquitous computing systems. We tested methods such as “Wizard of Oz” and design games in a real domain, the dental surgery, in an attempt to create a system that is: affordable; minimally disruptive of the natural flow of work; and improves human-computer interaction. In doing so we found that such activities allowed the practitioners to be on a ‘level playing ground’ with designers and engineers. The findings we present suggest that dentists are willing to engage in detailed exploration and constructive critique of technical design possibilities if the design ideas and prototypes are presented in the context of their work practice and are of a resolution and relevance that allow them to jointly explore and question with the design time. This paper is an extension of a short paper submitted to the Participatory Design Conference, 2004.

Research Monograph: Building Online Participatory Systems: Towards the Community Based Interactive Environmental Decision Support Systems

The importance of broadening community participation in environmental decision-making is widely recognized and lack of participation in this process appears to be a perennial problem. In this context, there have been calls from some academics for the more extensive use of geographic information systems (GIS) and distance learning technologies, accessible via the Internet, as a possible means to inform and empower communities. However, a number of problems exist. For instance, at present the scope for online interaction between policy-makers and citizens is currently limited. Contemporary web-based environmental information systems suffer from this lack of interactivity on the one hand and on the other hand from the apparent complexity for the lay user. This paper explores the issue of online community participation at the local level and attempts to construct a framework for a new (and potentially more effective) model of online participatory decision-making. The key components, system architecture and stages of such a model are introduced. This model, referred to as a ‘Community Based Interactive Environmental Decision Support System’, incorporates advanced information technologies, distance learning and community involvement tools which will be applied and evaluated in the field through a pilot project in Tokyo in the summer of 2002.

The entrepreneurial vlogger : participatory culture beyond the professional-amateur divide

In this chapter, we frame YouTube as an example of “co-creative” culture – whatever YouTube is, it is produced dynamically (that is, as an ongoing process, over time) as a result of many interconnected instances of participation, by many different people. In order to understand these co-creative relationships, it is important not to focus exclusively on how the “ordinary consumer” or “amateur producer,” are participating in YouTube; rather, we argue it is necessary to include the activities of “traditional media” companies and media professionals, and more importantly, the new models of media entrepreneurialism that are grounded in YouTube’s “grassroots” culture. Hence, this chapter focuses the role that “YouTube stars” – highly visible and successful “homegrown” performers and producers – play in modelling and negotiating these co-creative relationships within the context of YouTube’s social network; and the new models of entrepreneurship within participatory culture that they represent.

Digital storytelling as participatory public history in Australia

This chapter discusses digital storytelling as a methodology for participatory public history through a detailed reflection on an applied research project that integrated both public history and digital storytelling in the context of a new master-planned urban development: the Kelvin Grove Urban Village Sharing Stories project.

Encouraging ethics and challenging corruption : reforming governance in public institutions

Encouraging Ethics and Preventing Corruption brings theory and practice together in addressing the question: How are we to be ethical in public life and through public institutions? It is a major contribution to public sector ethics within Australia and internationally because it provides an exhaustive analysis of reform across a decade in one jurisdiction, Queensland, and then proceeds to itemise a best practice integrity system or ethics regime. Drawing on the extensive research of two of Australia's leading practical ethicists, this text is essential reading for all students and practitioners of applied and professional ethics in the public sphere. Part A of the text provides a preferred theoretical and conceptual framework which both justifies and guides the development of a public sector ethics regime. Part B examines the place of the individual within a world of institutional ethics. Part C outlines the Queensland governance reforms introduced since 1989 following the Fitzgerald Inquiry which exposed corruption in the police and ministry. The final chapter, the 'Epilogue', gathers the insights of earlier chapters and suggests a more explicitly ethics-centred approach to governance reform that may take us 'beyond best practice'. Clearly, while it is the Australian context we have in mind, we are confident that this is a text which addresses the quest for integrity and ethics in government wherever society is committed to social and liberal democratic ideals.

Fear, trust and Aborigines : The historical experience of state institutions and current encounters in the health system

Troubled dynamics between residents of an Aboriginal town in Queensland and the local health system were established during colonisation and consolidated during those periods of Australian history where the policies of 'protection' (segregation), integration and then assimilation held sway. The status of Aboriginal health is, in part, related to interactions between the residents' current and historical experiences of the health and criminal justice systems as together these agencies used medical and moral policing to legitimate dispossession, marginalisation, institutionalisation and control of the residents. The punitive regulations and ethnocentric strategies used by these institutions are within the living memory of many of the residents or in the published accounts of preceding generations. This paper explores current residents' memories and experiences.

Documentary practice in a participatory culture

Debates concerning the veracity, ethics and politics of the documentary form circle endlessly around the function of those who participate in it, and the meaning attributed to their participation. Great significance is attached to the way that documentary filmmakers do or do not participate in the world they seek to represent, just as great significance is attached to those subjects whose participation extends beyond playing the part of eyewitness or expert, such that they become part of the very filmmaking process itself. This Ph.D. explores the interface between documentary practice and participatory culture by looking at how their practices, discursive fields and histories intersect, but also by looking at how participating in one might mean participating in the other. In short, the research is an examination of participatory culture through the lens of documentary practice and documentary criticism. In the process, however, this examination of participatory culture will in turn shed light on documentary thinking, especially the meaning and function of ‘the participant’ in contemporary documentary practice. A number of ways of conceiving of participation in documentary practice are discussed in this research, but one of the ideas that gives purpose to that investigation is the notion that the participant in contemporary documentary practice is someone who belongs to a participatory culture in particular. Not only does this mean that those subjects who play a part in a documentary are already informed by their engagement with a range of everyday media practices before the documentary apparatus arrives, the audience for such films are similarly informed and engaged. This audience have their own expectations about how they should be addressed by media producers in general, a fact that feeds back into their expectations about participatory approaches to documentary practice too. It is the ambition of this research to get closer to understanding the relationship between participants in the audience, in documentary and ancillary media texts, as well as behind the camera, and to think about how these relationships constitute a context for the production and reception of documentary films, but also how this context might provide a model for thinking about participatory culture itself. One way that documentary practice and participatory culture converge in this research is in the kind of participatory documentary that I call the ‘Camera Movie’, a narrow mode of documentary filmmaking that appeals directly to contemporary audiences’ desires for innovation and participation, something that is achieved in this case by giving documentary subjects control of the camera. If there is a certain inevitability about this research having to contend with the notion of the ‘participatory documentary’, the ‘participatory camera’ also emerges strongly in this context, especially as a conduit between producer and consumer. Making up the creative component of this research are two documentaries about the reality television event Band In A Bubble, and participatory media practices more broadly. The single-screen film, Hubbub , gives form to the collective intelligence and polyphonous voice of contemporary audiences who must be addressed and solicited in increasingly innovative ways. One More Like That is a split-screen, DVD-Video with alternate audio channels selected by a user who thereby chooses who listens and who speaks in the ongoing conversation between media producers and media consumers. It should be clear from the description above that my own practice does not extend to highly interactive, multi-authored or web-enabled practices, nor the distributed practices one might associate with social media and online collaboration. Mine is fundamentally a single authored, documentary video practice that seeks to analyse and represent participatory culture on screen, and for this reason the Ph.D. refrains from a sustained discussion of the kinds of collaborative practices listed above. This is not to say that such practices don’t also represent an important intersection of documentary practice and participatory culture, they simply represent a different point of intersection. Being practice-led, this research takes its procedural cues from the nature of the practice itself, and sketches parameters that are most enabling of the idea that the practice sets the terms of its own investigation.

The challenges of participatory research with ‘tech-savvy’ youth

This paper focuses on participatory research and how it can be understood and employed when researching children and youth. The aim of this paper is to provide a theoretically and empirically grounded discussion of participatory research methodologies with respect to investigating the dynamic and evolving phenomenon of young people growing up in networked societies. Initially, we review the nature of participatory research and how other researchers have endeavoured to involve young people (children and youth) in their research projects. Our review of these approaches aims to elucidate what we see as recurring and emerging issues with respect to the methodological design of involving young people as co-researchers. In the light of these issues and in keeping with our aim, we offer a case study of our own research project that seeks to understand the ways in which high school students use new media and network ICT systems (Internet, mobile phone applications, social networking sites) to construct identities, form social relations, and engage in creative practices as part of their everyday lives. The article concludes by offering an assessment of our tripartite model of participatory research that may benefit other researchers who share a similar interest in youth and new media.

Web-based public participatory GIS

Decision support systems (DSS) have evolved rapidly during the last decade from stand alone or limited networked solutions to online participatory solutions. One of the major enablers of this change is the fastest growing areas of geographical information system (GIS) technology development that relates to the use of the Internet as a means to access, display, and analyze geospatial data remotely. World-wide many federal, state, and particularly local governments are designing to facilitate data sharing using interactive Internet map servers. This new generation DSS or planning support systems (PSS), interactive Internet map server, is the solution for delivering dynamic maps and GIS data and services via the world-wide Web, and providing public participatory GIS (PPGIS) opportunities to a wider community (Carver, 2001; Jankowski & Nyerges, 2001). It provides a highly scalable framework for GIS Web publishing, Web-based public participatory GIS (WPPGIS), which meets the needs of corporate intranets and demands of worldwide Internet access (Craig, 2002). The establishment of WPPGIS provides spatial data access through a support centre or a GIS portal to facilitate efficient access to and sharing of related geospatial data (Yigitcanlar, Baum, & Stimson, 2003). As more and more public and private entities adopt WPPGIS technology, the importance and complexity of facilitating geospatial data sharing is growing rapidly (Carver, 2003). Therefore, this article focuses on the online public participation dimension of the GIS technology. The article provides an overview of recent literature on GIS and WPPGIS, and includes a discussion on the potential use of these technologies in providing a democratic platform for the public in decision-making.

The Brisbane media map : participatory design and authentic learning to link students and industry

The Brisbane Media Map is both an online resource and a tertiary-level authentic learning project. The Brisbane Media Map is an online database which provides a detailed overview of about 600 media industry organisations in Brisbane, Australia. In addition to providing contact details and synopses for each organisation’s profile, the Brisbane Media Map also includes supplementary information on current issues, trends, and individuals in the media and communication industry sectors. This resource is produced and updated annually by final-year undergraduate Media and Communication students. This article introduces the Brisbane Media Map, its functionality and systems design approach, as well as its alignment with key learning infrastructures. It examines authentic learning as the pedagogical framework underpinning the ongoing development work of the resource and highlights some synergies of this framework with participatory design principles. The Brisbane Media Map is a useful example of an authentic learning approach that successfully engages students of non-traditional and non-design areas of study in human-computer interaction, usability, and participatory design activities.

An examination of the knowledge barriers in participatory design and the prospects for embedded research

Participatory design has the moral and pragmatic tenet of including those who will be most affected by a design into the design process. However, good participation is hard to achieve and results linking project success and degree of participation are inconsistent. Through three case studies examining some of the challenges that different properties of knowledge – novelty, difference, dependence – can impose on the participatory endeavour we examine some of the consequences to the participatory process of failing to bridge across knowledge boundaries – syntactic, semantic, and pragmatic. One pragmatic consequence, disrupting the user’s feeling of involvement to the project, has been suggested as a possible explanation for the inconsistent results linking participation and project success. To aid in addressing these issues a new form of participatory research, called embedded research, is proposed and examined within the framework of the case studies and knowledge framework with a call for future research into its possibilities.

Ourplace : the convergence of locative media and online participatory culture

The trans-locative potential of the Internet has driven the design of many online applications. Online communities largely cluster around topics of interest, which take precedence over participants’ geographical locations. The site of production is often disregarded when creative content appears online. However, for some, a sense of place is a defining aspect of creativity. Yet environments that focus on the display and sharing of regionally situated content have, so far, been largely overlooked. Recent developments in geo-technologies have precipitated the emergence of a new field of interactive media. Entitled locative media, it emphasizes the geographical context of media. This paper argues that we might combine practices of locative media (experiential mapping and geo-spatial annotation) with aspects of online participatory culture (uploading, file-sharing and search categorization) to produce online applications that support geographically ‘located’ communities. It discusses the design considerations and possibilities of this convergence,making reference to an example, OurPlace 3G to 3D, which has to date been developed as a prototype.1 It goes on to discuss the benefits and potential uses of such convergent applications, including the co-production of spatial- emporal narratives of place.