Crowdsourcing interactions : using crowdsourcing for evaluating interactive information retrieval systems

In the field of information retrieval (IR), researchers and practitioners are often faced with a demand for valid approaches to evaluate the performance of retrieval systems. The Cranfield experiment paradigm has been dominant for the in-vitro evaluation of IR systems. Alternative to this paradigm, laboratory-based user studies have been widely used to evaluate interactive information retrieval (IIR) systems, and at the same time investigate users’ information searching behaviours. Major drawbacks of laboratory-based user studies for evaluating IIR systems include the high monetary and temporal costs involved in setting up and running those experiments, the lack of heterogeneity amongst the user population and the limited scale of the experiments, which usually involve a relatively restricted set of users. In this paper, we propose an alternative experimental methodology to laboratory-based user studies. Our novel experimental methodology uses a crowdsourcing platform as a means of engaging study participants. Through crowdsourcing, our experimental methodology can capture user interactions and searching behaviours at a lower cost, with more data, and within a shorter period than traditional laboratory-based user studies, and therefore can be used to assess the performances of IIR systems. In this article, we show the characteristic differences of our approach with respect to traditional IIR experimental and evaluation procedures. We also perform a use case study comparing crowdsourcing-based evaluation with laboratory-based evaluation of IIR systems, which can serve as a tutorial for setting up crowdsourcing-based IIR evaluations.

Visual analytics for large-scale bioinformatic data sets

Rapid advances in sequencing technologies (Next Generation Sequencing or NGS) have led to a vast increase in the quantity of bioinformatics data available, with this increasing scale presenting enormous challenges to researchers seeking to identify complex interactions. This paper is concerned with the domain of transcriptional regulation, and the use of visualisation to identify relationships between specific regulatory proteins (the transcription factors or TFs) and their associated target genes (TGs). We present preliminary work from an ongoing study which aims to determine the effectiveness of different visual representations and large scale displays in supporting discovery. Following an iterative process of implementation and evaluation, representations were tested by potential users in the bioinformatics domain to determine their efficacy, and to understand better the range of ad hoc practices among bioinformatics literate users. Results from two rounds of small scale user studies are considered with initial findings suggesting that bioinformaticians require richly detailed views of TF data, features to compare TF layouts between organisms quickly, and ways to keep track of interesting data points.

My media studies : the history and future of ideas

In the context of a multi-paper special issue of TVNM on the future of media studies, this paper traces the tradition of ‘active audience’ theory in TV scholarship, arguing that it has much to offer in the study of new digital media, especially an approach to user-created content and dynamics of change. The paper argues for a ‘cultural science’ approach to ‘active audiences’ in order to analyse and understand how non-professionals and consumers contribute to the growth of knowledge in complex open media systems.

Empowerment games : empathic design probes for user participation in community housing

This paper documents the empowering process of a group of public housing residents through different design probing exercises. These exercises worked along with existing social processes without any involvement of designers. This paper shows how a design researcher devised a series of probing tools called "empowerment games" with a group of active users. These games are self-learning tools for making the abstract language of design legible to users. The main purpose of this intitiative was to change the preconception of govenmental bodies and professional designers of the passivity of the users with regard to their designed environment. This was the first case of the application of a participatory design process in Hong Kong subsidized housing. Design empathy is a central skill when working with users throughout the whole design research project.

From Prosumer to Produser: Understanding User-Led Content Creation

Alvin Toffler’s image of the prosumer (1970, 1980, 1990) continues to influence in a significant way our understanding of the user-led, collaborative processes of content creation which are today labelled “social media” or “Web 2.0”. A closer look at Toffler’s own description of his prosumer model reveals, however, that it remains firmly grounded in the mass media age: the prosumer is clearly not the self-motivated creative originator and developer of new content which can today be observed in projects ranging from open source software through Wikipedia to Second Life, but simply a particularly well-informed, and therefore both particularly critical and particularly active, consumer. The highly specialised, high end consumers which exist in areas such as hi-fi or car culture are far more representative of the ideal prosumer than the participants in non-commercial (or as yet non-commercial) collaborative projects. And to expect Toffler’s 1970s model of the prosumer to describe these 21st-century phenomena was always an unrealistic expectation, of course. To describe the creative and collaborative participation which today characterises user-led projects such as Wikipedia, terms such as ‘production’ and ‘consumption’ are no longer particularly useful – even in laboured constructions such as ‘commons-based peer-production’ (Benkler 2006) or ‘p2p production’ (Bauwens 2005). In the user communities participating in such forms of content creation, roles as consumers and users have long begun to be inextricably interwoven with those as producer and creator: users are always already also able to be producers of the shared information collection, regardless of whether they are aware of that fact – they have taken on a new, hybrid role which may be best described as that of a produser (Bruns 2008). Projects which build on such produsage can be found in areas from open source software development through citizen journalism to Wikipedia, and beyond this also in multi-user online computer games, filesharing, and even in communities collaborating on the design of material goods. While addressing a range of different challenges, they nonetheless build on a small number of universal key principles. This paper documents these principles and indicates the possible implications of this transition from production and prosumption to produsage.

User behaviours and intentions with digital news media in Australia

This paper summarises findings from a survey of user behaviors and intentions towards digital media and information in Australia. It was undertaken in the first quarter of 2009 by the Queensland University of Technology Creative Industries Faculty and was funded by the Smart Services Cooperative Research Centre. The survey targeted users of 2 news and information sites that are available online only. Findings highlighted differences between the 18-24 year age segment and older users. Social networks (specifically friends and family) were rated as the least reliable, relevant and accurate sources of news. Other findings indicate online news sources that are associated with an established newspaper are highly valued as reliable, relevant and accurate news sources by most people. While most people prefer to use online news sources, there is a great deal of variation in the ways in which people actually use online news. From a total of 524 respondents to the survey it was possible to identify three main types of online news consumers: convenience, loyal and customising users.

The user-led disruption : self-(re)broadcasting at Justin.tv and elsewhere

The rise of videosharing and self-(re)broadcasting Web services is posing new threats to a television industry already struggling with the impact of filesharing networks. This paper outlines these threats, focussing especially on the DIY re-broadcasting of live sports using Websites such as Justin.tv and a range of streaming media networks built on peer-to-peer filesharing technology.

User-Generated Content als Qualitätsmedium? Alternative Anreize für Qualitätscontent

Angetrieben und unterstützt durch Web-2.0-Technologien, gibt es heute einen Trend zur Verbindung der Nutzung und Produktion von Inhalten als Produtzung (engl. produsage ). Um dabei die Qualität der erstellten Inhalte und eine nachhaltige Teilnahme der Nutzer sicherzustellen, müsen vier grundlegende Prinzipien eingehalten werden: * Größtmögliche Offenheit. * Ankurbeln der Gemeinschaft durch Inhalte und Hilfsmittel. * Unterstützung der Gruppendynamik und Abtretung von Verantwortung. * Keine Ausbeutung der Gemeinschaft und ihrer Arbeit.

User behaviours and intentions in digital media in Australia

This paper reports findings from a study of user behaviours and intentions towards online news and information in Australia, undertaken by the Queensland University of Technology Creative Industries Faculty and the Smart Services Cooperative Research Centre. It has used a literature review, online survey, focus groups and interviews to explore attitudes and behaviours towards online news and information. The literature review on consumer user of online media highlighted emerging technical opportunities, and flagged existing barriers to access experienced by consumers in the Australian digital media sector. The literature review highlighted multiple disconnects between consumer interests in online news and their ability to fulfil them. This presents an opportunity for news entities to appraise and resolve. Doing so may enhance their service offering, attract consumers and improve loyalty. These themes were further explored by the survey. The survey results revealed three typologies of user, described as ‘convenience’, ‘loyal’ and ‘customising’. Convenience users tend to access news by default, for example when they log out of email. Loyal users seek out a trusted brand such as mainstream news mastheads. Customising users tend to tailor news to their preferences, and be the first to use leading edge media. Respondents to the survey were then invited to participate in focus groups, which aimed to test the survey results. Consumer perceptions and attitudes are important factors in progression towards an information economy, because ultimately consumers are customers. By segmenting the online news market according to customer typology, media providers may identify new opportunities to attract and retain customers.

Remote interaction and user research : anaesthetic preadmission activity

In the absence of telehealth technology, rural patients must travel to a regional or metropolitan hospital for a preadmission consultation one week before their surgery. Currently, examination of the patient’s chest using a stethoscope (auscultation) is not possible over a telehealth network as existing digital stethoscopes have been designed for in-person auscultation. We report on the initial phase of research which ultimately aims to design a digital stethoscope for use in the telehealth context. This initial research phase describes the complexity of the activity of preadmission clinics and the implications for the design of the stethoscope. The research is conducted through field studies of existing face-to-face and remote consultations.

User-Created Content and Online Social Networks

Traditional media are under assault from digital technologies. Online advertising is eroding the financial basis of newspapers and television, demarcations between different forms of media are fading, and audiences are fragmenting. We can podcast our favourite radio show, data accompanies television programs, and we catch up with newspaper stories on our laptops. Yet mainstream media remain enormously powerful. The Media and Communications in Australia offers a systematic introduction to this dynamic field. Fully updated and revised to take account of recent developments, this third edition outlines the key media industries and explains how communications technologies are impacting on them. It provides a thorough overview of the main approaches taken in studying the media, and includes new chapters on social media, gaming, telecommunications, sport and cultural diversity. With contributions from some of Australia's best researchers and teachers in the field, The Media and Communications in Australia is the most comprehensive and reliable introduction to media and communications available. It is an ideal student text, and a reference for teachers of media and anyone interested in this influential industry.

Asia and the Commons : case studies 2008

A collection of case studies of individuals and organisations utilising open models in the Asia Pacific and associated regions. The case studies represent activities in nine countries, broader regions such as the Arab nations, and global efforts towards sustainability and social justice, revealing creative ways of participating in the commons. Featured are remix artists, performers, open source software programmers, film makers, collecting institutions and publishing houses focused on democracy and change, who demonstrate a diverse set of motivations to engage with the shared ideals of openness and community collaboration.

New Media and Public Communication: Mapping Australian User-Created Content in Online Social Networks

Presentation describling a project in data intensive research in the humanities. Measuring activity of publically available data in social networks such as Blogosphere, Twitter, Flickr, YouTube

Reading nineteenth century schoolbooks online : user behavior in a digitized special collection

Special collections, because of the issues associated with conservation and use, a feature they share with archives, tend to be the most digitized areas in libraries. The Nineteenth Century Schoolbooks collection is a collection of 9000 rarely held nineteenth-century schoolbooks that were painstakingly collected over a lifetime of work by Prof. John A. Nietz, and donated to the Hillman Library at the University of Pittsburgh in 1958, which has since grown to 15,000. About 140 of these texts are completely digitized and showcased in a publicly accessible website through the University of Pittsburgh’s Library, along with a searchable bibliography of the entire collection, which expanded the awareness of this collection and its user base to beyond the academic community. The URL for the website is http://digital.library.pitt.edu/nietz/. The collection is a rich resource for researchers studying the intellectual, educational, and textbook publishing history of the United States. In this study, we examined several existing records collected by the Digital Research Library at the University of Pittsburgh in order to determine the identity and searching behaviors of the users of this collection. Some of the records examined include: 1) The results of a 3-month long user survey, 2) User access statistics including search queries for a period of one year, a year after the digitized collection became publicly available in 2001, and 3) E-mail input received by the website over 4 years from 2000-2004. The results of the study demonstrate the differences in online retrieval strategies used by academic researchers and historians, archivists, avocationists, and the general public, and the importance of facilitating the discovery of digitized special collections through the use of electronic finding aids and an interactive interface with detailed metadata.