Quality control, validation and user feedback of the European Flood Alert System (EFAS)

The quality control, validation and verification of the European Flood Alert System (EFAS) are described. EFAS is designed as a flood early warning system at pan-European scale, to complement national systems and provide flood warnings more than 2 days before a flood. On average 20–30 alerts per year are sent out to the EFAS partner network which consists of 24 National hydrological authorities responsible for transnational river basins. Quality control of the system includes the evaluation of the hits, misses and false alarms, showing that EFAS has more than 50% of the time hits. Furthermore, the skills of both the meteorological as well as the hydrological forecasts are evaluated, and are included here for a 10-year period. Next, end-user needs and feedback are systematically analysed. Suggested improvements, such as real-time river discharge updating, are currently implemented.

Overview of the INEX 2010 focused relevance feedback track

The INEX 2010 Focused Relevance Feedback track offered a refined approach to the evaluation of Focused Relevance Feedback algorithms through simulated exhaustive user feedback. As in traditional approaches we simulated a user-in-the loop by re-using the assessments of ad-hoc retrieval obtained from real users who assess focused ad-hoc retrieval submissions. The evaluation was extended in several ways: the use of exhaustive relevance feedback over entire runs; the evaluation of focused retrieval where both the retrieval results and the feedback are focused; the evaluation was performed over a closed set of documents and complete focused assessments; the evaluation was performed over executable implementations of relevance feedback algorithms; and �finally, the entire evaluation platform is reusable. We present the evaluation methodology, its implementation, and experimental results obtained for nine submissions from three participating organisations.

Overview of the INEX 2011 relevance feedback track

The INEX 2011 Relevance Feedback track offered a refined approach to the evaluation of Focused Relevance Feedback algorithms through simulated exhaustive user feedback. Run in largely identical fashion to the Relevance Feedback track in INEX 2010[2], we simulated a user-in-the loop by re-using the assessments of ad-hoc retrieval obtained from real users who assess focused ad-hoc retrieval submissions. We present the evaluation methodology, its implementation, and experimental results obtained for four submissions from two participating organisations. As the task and evaluation methods did not change between INEX 2010 and now, explanations of these details from the INEX 2010 version of the track have been repeated verbatim where appropriate.

Discovering user preferences by using time entries in click-through data to improve search engine results

The search engine log files have been used to gather direct user feedback on the relevancy of the documents presented in the results page. Typically the relative position of the clicks gathered from the log files is used a proxy for the direct user feedback. In this paper we identify reasons for the incompleteness of the relative position of clicks for deciphering the user preferences. Hence, we propose the use of time spent by the user in reading through the document as indicative of user preference for a document with respect to a query. Also, we identify the issues involved in using the time measure and propose means to address them.

User perspective on palliative care services: experiences of middle-aged partners bereaved through cancer known to social work services in Northern Ireland

This qualitative research study explores experiences of partners bereaved through cancer, who were resident in an urban area of Northern Ireland and who had been service users of the social work services. Data were collected in 2004 from 10 individuals who participated in semi-structured interviews. Emergent themes were identified using thematic content analysis and findings analysed under four categories: cancer journey; impact of bereavement; process of adjustment and change; and experience of support services. Opportunities to facilitate communication were not always maximised, often resulting in poor bereavement outcomes. Although hospices undertook bereavement risk assessment, participants were unaware of its use and queried its accuracy without service user involvement. The most cited informal support was family and friends, although such help was time-limited. Service user feedback regarding social workers was generally positive; however, there was a lack of knowledge about their role in palliative care. Post-bereavement adjustment was influenced by the quality of social networks, the responsibilities of lone parenthood, and challenges to life values and core beliefs. A framework for palliative care social work has been recommended based on research findings.

An investigation into the viability of LibraryThing for promotional and user engagement purposes in libraries

Structured abstract: Purpose: LibraryThing is a Web 2.0 tool allowing users to catalogue books using data drawn from sources such as Amazon and the Library of Congress and has facilities such as tagging and interest groups. This study evaluates whether LibraryThing is a valuable tool for libraries to use for promotional and user engagement purposes. Methodology: This study used a sequential mixed methods 3 phase design: (1) the identification of LibraryThing features for user engagement or promotional purposes, (2) exploratory semi-structured interviews (3) a questionnaire. Findings: Several uses of LibraryThing for promotional and user engagement purposes were identified. The most popular reason libraries used LibraryThing was to promote the library or library stock, with most respondents using it specifically to highlight collections of books. Monitoring of patron usage was low and many respondents had not received any feedback. LibraryThing was commonly reported as being easy to use, remotely accessible, and having low cost, whilst its main drawbacks were the 200 book limit for free accounts, and it being a third-party site. The majority of respondents felt LibraryThing was a useful tool for libraries. Practical implications: LibraryThing has most value as a promotional tool for libraries. Libraries should actively monitor patron usage of their LibraryThing account or request user feedback to ensure that LibraryThing provides a truly valuable service for their library. Orginality : There is little research on the value of LibraryThing for libraries, or librarians perceptions of LibraryThing as a Web 2.0 tool.

Semi-supervised learning for relevance feedback on image retrieval tasks

Relevance feedback approaches have been established as an important tool for interactive search, enabling users to express their needs. However, in view of the growth of multimedia collections available, the user efforts required by these methods tend to increase as well, demanding approaches for reducing the need of user interactions. In this context, this paper proposes a semi-supervised learning algorithm for relevance feedback to be used in image retrieval tasks. The proposed semi-supervised algorithm aims at using both supervised and unsupervised approaches simultaneously. While a supervised step is performed using the information collected from the user feedback, an unsupervised step exploits the intrinsic dataset structure, which is represented in terms of ranked lists of images. Several experiments were conducted for different image retrieval tasks involving shape, color, and texture descriptors and different datasets. The proposed approach was also evaluated on multimodal retrieval tasks, considering visual and textual descriptors. Experimental results demonstrate the effectiveness of the proposed approach.

SaveRF: Towards efficient relevance feedback search

In multimedia retrieval, a query is typically interactively refined towards the ‘optimal’ answers by exploiting user feedback. However, in existing work, in each iteration, the refined query is re-evaluated. This is not only inefficient but fails to exploit the answers that may be common between iterations. In this paper, we introduce a new approach called SaveRF (Save random accesses in Relevance Feedback) for iterative relevance feedback search. SaveRF predicts the potential candidates for the next iteration and maintains this small set for efficient sequential scan. By doing so, repeated candidate accesses can be saved, hence reducing the number of random accesses. In addition, efficient scan on the overlap before the search starts also tightens the search space with smaller pruning radius. We implemented SaveRF and our experimental study on real life data sets show that it can reduce the I/O cost significantly.

Recommending best answer in a collaborative question answering system

The World Wide Web has become a medium for people to share information. People use Web-based collaborative tools such as question answering (QA) portals, blogs/forums, email and instant messaging to acquire information and to form online-based communities. In an online QA portal, a user asks a question and other users can provide answers based on their knowledge, with the question usually being answered by many users. It can become overwhelming and/or time/resource consuming for a user to read all of the answers provided for a given question. Thus, there exists a need for a mechanism to rank the provided answers so users can focus on only reading good quality answers. The majority of online QA systems use user feedback to rank users’ answers and the user who asked the question can decide on the best answer. Other users who didn’t participate in answering the question can also vote to determine the best answer. However, ranking the best answer via this collaborative method is time consuming and requires an ongoing continuous involvement of users to provide the needed feedback. The objective of this research is to discover a way to recommend the best answer as part of a ranked list of answers for a posted question automatically, without the need for user feedback. The proposed approach combines both a non-content-based reputation method and a content-based method to solve the problem of recommending the best answer to the user who posted the question. The non-content method assigns a score to each user which reflects the users’ reputation level in using the QA portal system. Each user is assigned two types of non-content-based reputations cores: a local reputation score and a global reputation score. The local reputation score plays an important role in deciding the reputation level of a user for the category in which the question is asked. The global reputation score indicates the prestige of a user across all of the categories in the QA system. Due to the possibility of user cheating, such as awarding the best answer to a friend regardless of the answer quality, a content-based method for determining the quality of a given answer is proposed, alongside the non-content-based reputation method. Answers for a question from different users are compared with an ideal (or expert) answer using traditional Information Retrieval and Natural Language Processing techniques. Each answer provided for a question is assigned a content score according to how well it matched the ideal answer. To evaluate the performance of the proposed methods, each recommended best answer is compared with the best answer determined by one of the most popular link analysis methods, Hyperlink-Induced Topic Search (HITS). The proposed methods are able to yield high accuracy, as shown by correlation scores: Kendall correlation and Spearman correlation. The reputation method outperforms the HITS method in terms of recommending the best answer. The inclusion of the reputation score with the content score improves the overall performance, which is measured through the use of Top-n match scores.

New media services : ratings and recommendations websites in the travel and tourism industry

The travel and hospitality industry is one which relies especially crucially on word of mouth, both at the level of overall destinations (Australia, Queensland, Brisbane) and at the level of travellers’ individual choices of hotels, restaurants, sights during their trips. The provision of such word-of-mouth information has been revolutionised over the past decade by the rise of community-based Websites which allow their users to share information about their past and future trips and advise one another on what to do or what to avoid during their travels. Indeed, the impact of such user-generated reviews, ratings, and recommendations sites has been such that established commercial travel advisory publishers such as Lonely Planet have experienced a pronounced downturn in sales ¬– unless they have managed to develop their own ways of incorporating user feedback and contributions into their publications. This report examines the overall significance of ratings and recommendation sites to the travel industry, and explores the community, structural, and business models of a selection of relevant ratings and recommendations sites. We identify a range of approaches which are appropriate to the respective target markets and business aims of these organisations, and conclude that there remain significant opportunities for further operators especially if they aim to cater for communities which are not yet appropriately served by specific existing sites. Additionally, we also point to the increasing importance of connecting stand-alone ratings and recommendations sites with general social media spaces like Facebook, Twitter, and LinkedIn, and of providing mobile interfaces which enable users to provide updates and ratings directly from the locations they happen to be visiting. In this report, we profile the following sites: * TripAdvisor, the international market leader for travel ratings and recommendations sites, with a membership of some 11 million users; * IgoUgo, the other leading site in this field, which aims to distinguish itself from the market leader by emphasising the quality of its content; * Zagat, a long-established publisher of restaurant guides which has translated its crowdsourcing model from the offline to the online world; * Lonely Planet’s Thorn Tree site, which attempts to respond to the rise of these travel communities by similarly harnessing user-generated content; * Stayz, which attempts to enhance its accommodation search and booking services by incorporating ratings and reviews functionality; and * BigVillage, an Australian-based site attempting to cater for a particularly discerning niche of travellers; * Dopplr, which connects travel and social networking in a bid to pursue the lucrative market of frequent and business travellers; * Foursquare, which builds on its mobile application to generate a steady stream of ‘check-ins’ and recommendations for hospitality and other services around the world; * Suite 101, which uses a revenue-sharing model to encourage freelance writers to contribute travel writing (amongst other genres of writing); * Yelp, the global leader in general user-generated product review and recommendation services. In combination, these profiles provide an overview of current developments in the travel ratings and recommendations space (and beyond), and offer an outlook for further possibilities. While no doubt affected by the global financial downturn and the reduction in travel that it has caused, travel ratings and recommendations remain important – perhaps even more so if a reduction in disposable income has resulted in consumers becoming more critical and discerning. The aggregated word of mouth from many tens of thousands of travellers which these sites provide certainly has a substantial influence on their users. Using these sites to research travel options has now become an activity which has spread well beyond the digirati. The same is true also for many other consumer industries, especially where there is a significant variety of different products available – and so, this report may also be read as a case study whose findings are able to be translated, mutatis mutandis, to purchasing decisions from household goods through consumer electronics to automobiles.

Design from the everyday: continuously evolving, embedded exploratory prototypes

One of the major challenges in the design of social technologies is the evaluation of their qualities of use and how they are appropriated over time. While the field of HCI abounds in short-term exploratory design and studies of use, relatively little attention has focused on the continuous development of prototypes longitudinally and studies of their emergent use. We ground the exploration and analysis of use in the everyday world, embracing contingency and open-ended use, through the use of a continuously-available exploratory prototype. Through examining use longitudinally, clearer insight can be gained of realistic, non-novelty usage and appropriation into everyday use. This paper sketches out a framework for design that puts a premium on immediate use and evolving the design in response to use and user feedback. While such design practices with continuously developing systems are common in the design of social technologies, they are little documented. We describe our approach and reflect upon its key characteristics, based on our experiences from two case studies. We also present five major patterns of long-term usage which we found useful for design.

Reframing the development of corporate learning

Due to increasing recognition by industry that partnerships with universities can lead to more effective knowledge and skills acquisition and deployment, corporate learning programmes are currently experiencing a resurgence of interest. Rethinking of corporations’ approaches to what has traditionally been classed as ‘training’ has resulted in a new focus on learning and the adoption of philosophies that underlie the academic paradigm. This paper reports on two studies of collaboration between major international engineering corporations and an Australian university, the aim of which was to up-skill the workforce in response to changing markets. The paper highlights the differences between the models of learning adopted in such collaboration and those in more conventional, university-based environments. The learning programmes combine the ADDIE (analysis, design, develop, implement and evaluate) development and workplace learning models. Adaptations that have added value for industry partners and recommendations as to how these can be evolved to cope with change are discussed. The learning is contextualised by industry- based subject matter experts working in close collaboration with university experts and learning designers to develop programmes that are reflective of current and future needs in the organisation. Results derived from user feedback indicate that the learning programmes are effectively aligned with the needs of the industry partners whilst simultaneously upholding academic ideals. In other words, it is possible to combine academic and more traditional approaches to develop corporate learning programmes that satisfy requirements in the workplace. Emerging from the study, a new conceptual framework for the development of corporate learning is presented.

Fixing the city one photo at a time : mobile logging of maintenance requests

We have designed a mobile application that takes advantage of the built-in features of smart phones such as camera and GPS that allow users to take geo-tagged photos while on the move. Urban residents can take pictures of broken street furniture and public property requiring repair, attach a brief description, and submit the information as a maintenance request to the local government organisation of their city. This paper discusses the design approach that led to the application, highlights a built-in mechanism to elicit user feedback, and evaluates the progress to date with user feedback and log statistics. It concludes with an outlook highlighting user requested features and our own design aspirations for moving from a reporting tool to a civic engagement tool.