Integrating understandability in the evaluation of consumer health search engines

In this paper we propose a method that integrates the no- tion of understandability, as a factor of document relevance, into the evaluation of information retrieval systems for con- sumer health search. We consider the gain-discount evaluation framework (RBP, nDCG, ERR) and propose two understandability-based variants (uRBP) of rank biased precision, characterised by an estimation of understandability based on document readability and by different models of how readability influences user understanding of document content. The proposed uRBP measures are empirically contrasted to RBP by comparing system rankings obtained with each measure. The findings suggest that considering understandability along with topicality in the evaluation of in- formation retrieval systems lead to different claims about systems effectiveness than considering topicality alone.

Perceptions of document relevance

This article presents a study of how humans perceive and judge the relevance of documents. Humans are adept at making reasonably robust and quick decisions about what information is relevant to them, despite the ever increasing complexity and volume of their surrounding information environment. The literature on document relevance has identified various dimensions of relevance (e.g., topicality, novelty, etc.), however little is understood about how these dimensions may interact. We performed a crowdsourced study of how human subjects judge two relevance dimensions in relation to document snippets retrieved from an internet search engine. The order of the judgment was controlled. For those judgments exhibiting an order effect, a q–test was performed to determine whether the order effects can be explained by a quantum decision model based on incompatible decision perspectives. Some evidence of incompatibility was found which suggests incompatible decision perspectives is appropriate for explaining interacting dimensions of relevance in such instances.

News via Voldemort: Parody accounts in topical discussions on Twitter

Parody accounts are prevalent on Twitter, offering irreverent interpretations of public figures, fictional characters and more. These accounts post comments framed within the context of their fictional universes or stereotypes of their subjects, responding in-character to topical events. This article positions parody accounts as a ritualised social media practice, an extension of fan practices and irreverent Internet culture. By providing a typology of parody accounts and analysing the topicality of selected parody accounts’ tweets, the research examines how these accounts contribute to topical discussions. In-character framing of topical comments allows parody accounts to offer original interpretations of breaking news that receive more attention than their other tweets. The presence and longevity of parody accounts underline the importance of humour on social media, including within news and topical coverage.