Overview of the INEX 2011 relevance feedback track

The INEX 2011 Relevance Feedback track offered a refined approach to the evaluation of Focused Relevance Feedback algorithms through simulated exhaustive user feedback. Run in largely identical fashion to the Relevance Feedback track in INEX 2010[2], we simulated a user-in-the loop by re-using the assessments of ad-hoc retrieval obtained from real users who assess focused ad-hoc retrieval submissions. We present the evaluation methodology, its implementation, and experimental results obtained for four submissions from two participating organisations. As the task and evaluation methods did not change between INEX 2010 and now, explanations of these details from the INEX 2010 version of the track have been repeated verbatim where appropriate.

The effectiveness of energy feedback for conservation and peak demand : a literature review

This paper reviews electricity consumption feedback literature to explore the potential of electricity feedback to affect residential consumers’ electricity usage patterns. The review highlights a substantial amount of literature covering the debate over the effectiveness of different feedback criteria to residential customer acceptance and overall conservation and peak demand reduction. Researchers studying the effects of feedback on everyday energy use have observed substantial variation in effect size, both within and between studies. Although researchers still continue to question the types of feedback that are most effective in encouraging conservation and peak load reduction, some trends have emerged. These include that feedback be received as quickly as possible to the time of consumption; be related to a standard; be clear and meaningful and where possible both direct and indirect feedback be customised to the customer. In general, the literature finds that feedback can reduce electricity consumption in homes by 5 to 20 per cent, but that significant gaps remain in our knowledge of the effectiveness and cost benefit of feedback.

The OO files : providing online formative feedback on Contract Law learning in a narrative setting

Feedback, both formative and summative, enables students to reflect on their understandings and to restructure their thinking to develop their capabilities. It can also encourage positive motivation and help boost self-esteem. Online multiple choice questions can be an efficient and effective means of providing timely formative feedback. At the same time, locating learning in a narrative environment can facilitate engaging and effective learning experiences. Narratives can help learners to navigate through information and support cognitive and imaginative engagement. This article will discuss The 00 Files, an online suite of modules containing multiple choice questions situated in the narrative of a fictional law firm. It notes student responses to the program and discusses lessons that may be learnt from its development which may be of assistance to academics considering the development of similar programs for their courses.

Stigmergy within web modelling languages : positive feedback mechanisms

Stigmergy is a biological term originally used when discussing insect or swarm behaviour, and describes a model supporting environment-based communication separating artefacts from agents. This phenomenon is demonstrated in the behavior of ants and their food foraging supported by pheromone trails, or similarly termites and their termite nest building process. What is interesting with this mechanism is that highly organized societies are formed without an apparent central management function. We see design features in Web sites that mimic stigmergic mechanisms as part of the User Interface and we have created generalizations of these patterns. Software development and Web site development techniques have evolved significantly over the past 20 years. Recent progress in this area proposes languages to model web applications to facilitate the nuances specific to these developments. These modeling languages provide a suitable framework for building reusable components encapsulating our design patterns of stigmergy. We hypothesize that incorporating stigmergy as a separate feature of a site’s primary function will ultimately lead to enhanced user coordination.

Feedback consistencies and inconsistencies : eight mentors’ observations on one preservice teacher’s lesson

Mentors play a key role in developing preservice teachers for their chosen careers and providing feedback appears as a significant relational interaction between the mentor and mentee that assists in guiding the mentee’s practices. Yet, what are mentors’ perspectives on providing feedback to their mentees? In this case study, eight mentors viewed a professional video recorded science lesson facilitated by a final-year preservice teacher during practicum for the purposes of providing oral feedback in a simulated mentor-mentee discussion. Findings showed that mentors’ feedback was variable in both their positive feedback and constructive criticisms and, in one case, the feedback was contrasting in nature. Implications are discussed, including preservice teachers receiving feedback from more than one mentor and universities researching the design of valid and reliable tools to guide mentors’ oral feedback.

Using formative feedback to identify and support first year chemistry students with missing or misconceptions. A Practice Report

Students entering tertiary studies possess a diverse range of prior experiences in their academic preparation for tertiary chemistry so academics need tools to enable them to respond to issues in diversity in conceptual models possessed by entering students. Concept inventories can be used to provide formative feedback to help students identify concepts that they need to address to improve construction of subsequent understanding enabling their learning. Modular, formative learning activities that can be administered inside or outside of class in first year chemistry courses have been developed. These activities address key missing and mis-conceptions possessed by incoming student. Engagement in these learning activities by students and academics will help shift the culture of diagnostic and formative assessment within the tertiary context and address issues around the secondary/tertiary transition. This diagnostic/intervention framework is currently being trialed across five Australian tertiary institutions encompassing a large heterogeneous sample of students.

Towards engaged consumption : new sources of inspiration for eco-feedback design

Eco-feedback interventions are capable of producing reductions in household energy consumption. Yet less is known about exactly how this reduction is achieved, how to maximise user engagement, or how to effectively translate engagement into energy saving. This paper discusses design opportunities for eco-feedback systems through observations of domestic energy use in both Western and rural developing world contexts. Drawing on case studies from these two contexts including 21 empirical interviews, we present an alternative framework for human-resource interaction, highlighting design opportunities for a transition towards more engaged and sustainable energy consumption among users.

Effect of the mediator in feedback mode-based SECM interrogation of indium tin-oxide and boron-doped diamond electrodes

Indium tin-oxide (ITO) and polycrystalline boron-doped diamond (BDD) have been examined in detail using the scanning electrochemical microscopy technique in feedback mode. For the interrogation of electrodes made from these materials, the choice of mediator has been varied. Using Ru(CN) 4− 6 (aq), ferrocene methanol (FcMeOH), Fe(CN) 3− 6 (aq) and Ru(NH 3) 3+ 6 (aq), approach curve experiments have been performed, and for purposes of comparison, calculations of the apparent heterogeneous electron transfer rates (k app) have been made using these data. In general, it would appear that values of k app are affected mainly by the position of the mediator reversible potential relative to the relevant semiconductor band edge (associated with majority carriers). For both the ITO (n type) and BDD (p type) electrodes, charge transfer is impeded and values are very low when using FcMeOH and Fe(CN) 3− 6 (aq) as mediators, and the use of Ru(NH 3) 3+ 6(aq) results in the largest value of k app. With ITO, the surface is chemically homogeneous and no variation is observed for any given mediator. Data is also presented where the potential of the ITO electrode is fixed using a ratio of the mediators Fe(CN) 3− 6(aq) and Fe(CN) 4− 6(aq). In stark contrast, the BDD electrode is quite the opposite and a range of k app values are observed for all mediators depending on the position on the surface. Both electrode surfaces are very flat and very smooth, and hence, for BDD, variations in feedback current imply a variation in the electrochemical activity. A comparison of the feedback current where the substrate is biased and unbiased shows a surprising degree of proportionality.

Curiosity to cupboard : self reported disengagement with energy use feedback over time

This paper discusses findings made during a study of energy use feedback in the home (eco-feedback), well after the novelty has worn off. Contributing towards four important knowledge gaps in the research, we explore eco-feedback over longer time scales, focusing on instances where the feedback was not of lasting benefit to users rather than when it was. Drawing from 23 semi-structured interviews with Australian householders, we found that an initially high level of engagement gave way over time to disinterest, neglect and in certain cases, technical malfunction. Additionally, preconceptions concerned with the “purpose” of the feedback were found to affect use. We propose expanding the scope of enquiry for eco-feedback in several ways, and describe how eco-feedback that better supports decision-making in the “maintenance phase”, i.e. once the initial novelty has worn off, may be key to longer term engagement.

Learner-focused evaluation cycles : facilitating learning using feedforward, concurrent and feedback evaluation

There is a growing trend to offer students learning opportunities that are flexible, innovative and engaging. As educators embrace student-centred agile teaching and learning methodologies, which require continuous reflection and adaptation, the need to evaluate students’ learning in a timely manner has become more pressing. Conventional evaluation surveys currently dominate the evaluation landscape internationally, despite recognition that they are insufficient to effectively evaluate curriculum and teaching quality. Surveys often: (1) fail to address the issues for which educators need feedback, (2) constrain student voice, (3) have low response rates and (4) occur too late to benefit current students. Consequently, this paper explores principles of effective feedback to propose a framework for learner-focused evaluation. We apply a three-stage control model, involving feedforward, concurrent and feedback evaluation, to investigate the intersection of assessment and evaluation in agile learning environments. We conclude that learner-focused evaluation cycles can be used to guide action so that evaluation is not undertaken simply for the benefit of future offerings, but rather to benefit current students by allowing ‘real-time’ learning activities to be adapted in the moment. As a result, students become co-producers of learning and evaluation becomes a meaningful, responsive dialogue between students and their instructors.

A reciprocal collaborative method using relevance feedback and feature importance

In a people-to-people matching systems, filtering is widely applied to find the most suitable matches. The results returned are either too many or only a few when the search is generic or specific respectively. The use of a sophisticated recommendation approach becomes necessary. Traditionally, the object of recommendation is the item which is inanimate. In online dating systems, reciprocal recommendation is required to suggest a partner only when the user and the recommended candidate both are satisfied. In this paper, an innovative reciprocal collaborative method is developed based on the idea of similarity and common neighbors, utilizing the information of relevance feedback and feature importance. Extensive experiments are carried out using data gathered from a real online dating service. Compared to benchmarking methods, our results show the proposed method can achieve noticeable better performance.

Objectifying the Subjective : Assessment and Feedback in Creative Arts Studio Learning and Teaching

This paper examines the Assessment and Feedback aspects of Studio Teaching as Creative Arts pedagogy. Prompted by USQ’s newly offered Bachelor of Creative Arts (BCA), the author has developed an Assessment Matrix specifically designed to satisfy a number of imperatives, including: • ‘objectifying’ the subjective aspects of creative practice as assessable coursework/research • providing the means by which accurate, detailed, personalised and confidential feedback may be provided to students individually • providing consistent, accurate, meaningful assessment records for student, lecturer, and institution • ensuring consistency, continuity, and transparency of assessment processes and records to satisfy quality audits • minimising marking and assessment time, whilst maximising assessment integrity and depth • requiring only basic level skills and knowledge of a computer application already in common use (Microsoft Excel) • adaptability to a range of creative courses ‐ across disciplines This Assessment Matrix has been in development (and trialled) since January 2009.

Motivating domestic energy conservation through comparative feedback in mobile applications and social networking sites

The progress of technology has led to the increased adoption of energy monitors among household energy consumers. While the monitors available on the market deliver real-time energy usage feedback to the consumer, the form of this data is usually unengaging and mundane. Moreover, it fails to address consumers with different motivations and needs to save and compare energy. This master‟s thesis project presents a study that seeks to inform design guidelines for differently motivated energy consumers. The focus of the research is on comparative feedback supported by a community of energy consumers. In particular, the discussed comparative feedback types are explanatory comparison, temporal self-comparison, norm comparison, one-on-one comparison and ranking, whereby the last three support exploring the potential of socialising energy-related feedback in social networking sites, such as Facebook. These feedback types were integrated in EnergyWiz – a mobile application that enables users to compare with their past performance, neighbours, contacts from social networking sites and other EnergyWiz users. The application was developed through a theory-driven approach and evaluated in personal, semi-structured interviews which provided insights on how motivation-related comparative feedback should be designed. It was also employed in expert focus group discussions which resulted in defining opportunities and challenges before mobile, social energy monitors. The findings have unequivocally shown that users with different motivations to compare and to conserve energy have different preferences for comparative feedback types and design. It was established that one of the most influential factors determining design factors is the people users compare to. In addition, the research found that even simple communication strategies in Facebook, such as wall posts and groups can contribute to engagement with energy conservation practices. The concept of mobility of the application was evaluated as positive since it provides place and time-independent access to the energy consumption data.

Q-FE: QPASTT's Whole of Organisation Client Feedback Framework

This report describes the development of a whole of organization framework for obtaining client feedback for the Queensland Program of Assistance to Survivors of Torture and Trauma (QPASTT)