A query-basis approach to parametrizing novelty-biased cumulative gain

Novelty-biased cumulative gain (α-NDCG) has become the de facto measure within the information retrieval (IR) community for evaluating retrieval systems in the context of sub-topic retrieval. Setting the incorrect value of parameter α in α-NDCG prevents the measure from behaving as desired in particular circumstances. In fact, when α is set according to common practice (i.e. α = 0.5), the measure favours systems that promote redundant relevant sub-topics rather than provide novel relevant ones. Recognising this characteristic of the measure is important because it affects the comparison and the ranking of retrieval systems. We propose an approach to overcome this problem by defining a safe threshold for the value of α on a query basis. Moreover, we study its impact on system rankings through a comprehensive simulation.

Investigating subspace distances in semantic spaces

Semantic space models of word meaning derived from co-occurrence statistics within a corpus of documents, such as the Hyperspace Analogous to Language (HAL) model, have been proposed in the past. While word similarity can be computed using these models, it is not clear how semantic spaces derived from different sets of documents can be compared. In this paper, we focus on this problem, and we revisit the proposal of using semantic subspace distance measurements [1]. In particular, we outline the research questions that still need to be addressed to investigate and validate these distance measures. Then, we describe our plans for future research.

An analysis of ranking principles and retrieval strategies

The assumptions underlying the Probability Ranking Principle (PRP) have led to a number of alternative approaches that cater or compensate for the PRP’s limitations. All alternatives deviate from the PRP by incorporating dependencies. This results in a re-ranking that promotes or demotes documents depending upon their relationship with the documents that have been already ranked. In this paper, we compare and contrast the behaviour of state-of-the-art ranking strategies and principles. To do so, we tease out analytical relationships between the ranking approaches and we investigate the document kinematics to visualise the effects of the different approaches on document ranking.

When two is better than one : a study of ranking paradigms and their integrations for subtopic retrieval

In this paper, we consider the problem of document ranking in a non-traditional retrieval task, called subtopic retrieval. This task involves promoting relevant documents that cover many subtopics of a query at early ranks, providing thus diversity within the ranking. In the past years, several approaches have been proposed to diversify retrieval results. These approaches can be classified into two main paradigms, depending upon how the ranks of documents are revised for promoting diversity. In the first approach subtopic diversification is achieved implicitly, by choosing documents that are different from each other, while in the second approach this is done explicitly, by estimating the subtopics covered by documents. Within this context, we compare methods belonging to the two paradigms. Furthermore, we investigate possible strategies for integrating the two paradigms with the aim of formulating a new ranking method for subtopic retrieval. We conduct a number of experiments to empirically validate and contrast the state-of-the-art approaches as well as instantiations of our integration approach. The results show that the integration approach outperforms state-of-the-art strategies with respect to a number of measures.

University of Glasgow at ImageCLEFPhoto 2009 : optimising similarity and diversity in image retrieval

In this paper we describe the approaches adopted to generate the runs submitted to ImageCLEFPhoto 2009 with an aim to promote document diversity in the rankings. Four of our runs are text based approaches that employ textual statistics extracted from the captions of images, i.e. MMR [1] as a state of the art method for result diversification, two approaches that combine relevance information and clustering techniques, and an instantiation of Quantum Probability Ranking Principle. The fifth run exploits visual features of the provided images to re-rank the initial results by means of Factor Analysis. The results reveal that our methods based on only text captions consistently improve the performance of the respective baselines, while the approach that combines visual features with textual statistics shows lower levels of improvements.

The Quantum Probability Ranking Principle for Information Retrieval

While the Probability Ranking Principle for Information Retrieval provides the basis for formal models, it makes a very strong assumption regarding the dependence between documents. However, it has been observed that in real situations this assumption does not always hold. In this paper we propose a reformulation of the Probability Ranking Principle based on quantum theory. Quantum probability theory naturally includes interference effects between events. We posit that this interference captures the dependency between the judgement of document relevance. The outcome is a more sophisticated principle, the Quantum Probability Ranking Principle, that provides a more sensitive ranking which caters for interference/dependence between documents’ relevance.

Semantic spaces : measuring the distance between different subspaces

Semantic Space models, which provide a numerical representation of words’ meaning extracted from corpus of documents, have been formalized in terms of Hermitian operators over real valued Hilbert spaces by Bruza et al. [1]. The collapse of a word into a particular meaning has been investigated applying the notion of quantum collapse of superpositional states [2]. While the semantic association between words in a Semantic Space can be computed by means of the Minkowski distance [3] or the cosine of the angle between the vector representation of each pair of words, a new procedure is needed in order to establish relations between two or more Semantic Spaces. We address the question: how can the distance between different Semantic Spaces be computed? By representing each Semantic Space as a subspace of a more general Hilbert space, the relationship between Semantic Spaces can be computed by means of the subspace distance. Such distance needs to take into account the difference in the dimensions between subspaces. The availability of a distance for comparing different Semantic Subspaces would enable to achieve a deeper understanding about the geometry of Semantic Spaces which would possibly translate into better effectiveness in Information Retrieval tasks.

Why assessing relevance in medical IR is demanding

This study investigates if and why assessing relevance of clinical records for a clinical retrieval task is cognitively demanding. Previous research has highlighted the challenges and issues information retrieval systems are faced with when determining the relevance of documents in this domain, e.g., the vocabulary mismatch problem. Determining if this assessment imposes cognitive load on human assessors, and why this is the case, may shed lights on what are the (cognitive) processes that assessors use for determining document relevance (in this domain). High cognitive load may impair the ability of the user to make accurate relevance judgements and hence the design of IR mechanisms may need to take this into account in order to reduce the load.

Integrating understandability in the evaluation of consumer health search engines

In this paper we propose a method that integrates the no- tion of understandability, as a factor of document relevance, into the evaluation of information retrieval systems for con- sumer health search. We consider the gain-discount evaluation framework (RBP, nDCG, ERR) and propose two understandability-based variants (uRBP) of rank biased precision, characterised by an estimation of understandability based on document readability and by different models of how readability influences user understanding of document content. The proposed uRBP measures are empirically contrasted to RBP by comparing system rankings obtained with each measure. The findings suggest that considering understandability along with topicality in the evaluation of in- formation retrieval systems lead to different claims about systems effectiveness than considering topicality alone.

Overview of the ShARe/CLEF eHealth Evaluation Lab 2014

This paper reports on the 2nd ShARe/CLEFeHealth evaluation lab which continues our evaluation resource building activities for the medical domain. In this lab we focus on patients' information needs as opposed to the more common campaign focus of the specialised information needs of physicians and other healthcare workers. The usage scenario of the lab is to ease patients and next-of-kins' ease in understanding eHealth information, in particular clinical reports. The 1st ShARe/CLEFeHealth evaluation lab was held in 2013. This lab consisted of three tasks. Task 1 focused on named entity recognition and normalization of disorders; Task 2 on normalization of acronyms/abbreviations; and Task 3 on information retrieval to address questions patients may have when reading clinical reports. This year's lab introduces a new challenge in Task 1 on visual-interactive search and exploration of eHealth data. Its aim is to help patients (or their next-of-kin) in readability issues related to their hospital discharge documents and related information search on the Internet. Task 2 then continues the information extraction work of the 2013 lab, specifically focusing on disorder attribute identification and normalization from clinical text. Finally, this year's Task 3 further extends the 2013 information retrieval task, by cleaning the 2013 document collection and introducing a new query generation method and multilingual queries. De-identified clinical reports used by the three tasks were from US intensive care and originated from the MIMIC II database. Other text documents for Tasks 1 and 3 were from the Internet and originated from the Khresmoi project. Task 2 annotations originated from the ShARe annotations. For Tasks 1 and 3, new annotations, queries, and relevance assessments were created. 50, 79, and 91 people registered their interest in Tasks 1, 2, and 3, respectively. 24 unique teams participated with 1, 10, and 14 teams in Tasks 1, 2 and 3, respectively. The teams were from Africa, Asia, Canada, Europe, and North America. The Task 1 submission, reviewed by 5 expert peers, related to the task evaluation category of Effective use of interaction and targeted the needs of both expert and novice users. The best system had an Accuracy of 0.868 in Task 2a, an F1-score of 0.576 in Task 2b, and Precision at 10 (P@10) of 0.756 in Task 3. The results demonstrate the substantial community interest and capabilities of these systems in making clinical reports easier to understand for patients. The organisers have made data and tools available for future research and development.

An analytical framework for quantifying aquifer response time scales associated with transient boundary conditions

A major challenge in studying coupled groundwater and surface-water interactions arises from the considerable difference in the response time scales of groundwater and surface-water systems affected by external forcings. Although coupled models representing the interaction of groundwater and surface-water systems have been studied for over a century, most have focused on groundwater quantity or quality issues rather than response time. In this study, we present an analytical framework, based on the concept of mean action time (MAT), to estimate the time scale required for groundwater systems to respond to changes in surface-water conditions. MAT can be used to estimate the transient response time scale by analyzing the governing mathematical model. This framework does not require any form of transient solution (either numerical or analytical) to the governing equation, yet it provides a closed form mathematical relationship for the response time as a function of the aquifer geometry, boundary conditions, and flow parameters. Our analysis indicates that aquifer systems have three fundamental time scales: (i) a time scale that depends on the intrinsic properties of the aquifer; (ii) a time scale that depends on the intrinsic properties of the boundary condition, and; (iii) a time scale that depends on the properties of the entire system. We discuss two practical scenarios where MAT estimates provide useful insights and we test the MAT predictions using new laboratory-scale experimental data sets.

Using the Event Analysis of Systemic Teamwork (EAST) to explore conflict between different road user groups when making right hand turns at urban intersections

Collisions between different types of road users at intersections form a substantial component of the road toll. This paper presents an analysis of driver, cyclist, motorcyclist and pedestrian behaviour at intersections that involved the application of an integrated suite of ergonomics methods, the Event Analysis of Systemic Teamwork (EAST) framework, to on-road study data. EAST was used to analyse behaviour at three intersections using data derived from an on-road study of driver, cyclist, motorcyclist and pedestrian behaviour. The analysis shows the differences in behaviour and cognition across the different road user groups and pinpoints instances where this may be creating conflicts between different road users. The role of intersection design in creating these differences in behaviour and resulting conflicts is discussed. It is concluded that currently intersections are not designed in a way that supports behaviour across the four forms of road user studied. Interventions designed to improve intersection safety are discussed.

Intravitreal drug delivery in retinal disease : are we out of our depth?

Introduction With the ever-increasing global burden of retinal disease, there is an urgent need to vastly improve formulation strategies that enhance posterior eye delivery of therapeutics. Despite intravitreal administration having demonstrated notable superiority over other routes in enhancing retinal drug availability, there still exist various significant physical/biochemical barriers preventing optimal drug delivery into the retina. A further complication lies with an inability to reliably translate laboratory-based retinal models into a clinical setting. Several formulation approaches have recently been evaluated to improve intravitreal therapeutic outcomes, and our aim in this review is to highlight strategies that hold the most promise. Areas covered We discuss the complex barriers faced by the intravitreal route and examine how formulation strategies including implants, nanoparticulate carriers, viral vectors and sonotherapy have been utilized to attain both sustained delivery and enhanced penetration through to the retina. We conclude by highlighting the advances and limitations of current in vitro, ex vivo and in vivo retinal models in use by researchers globally. Expert opinion Various nanoparticle compositions have demonstrated the ability to overcome the retinal barriers successfully; however, their utility is limited to the laboratory setting. Optimization of these formulations and the development of more robust experimental retinal models are necessary to translate success in the laboratory into clinically efficacious outcomes.

Factors influencing robustness and effectiveness of conditional random fields in active learning frameworks

Active learning approaches reduce the annotation cost required by traditional supervised approaches to reach the same effectiveness by actively selecting informative instances during the learning phase. However, effectiveness and robustness of the learnt models are influenced by a number of factors. In this paper we investigate the factors that affect the effectiveness, more specifically in terms of stability and robustness, of active learning models built using conditional random fields (CRFs) for information extraction applications. Stability, defined as a small variation of performance when small variation of the training data or a small variation of the parameters occur, is a major issue for machine learning models, but even more so in the active learning framework which aims to minimise the amount of training data required. The factors we investigate are a) the choice of incremental vs. standard active learning, b) the feature set used as a representation of the text (i.e., morphological features, syntactic features, or semantic features) and c) Gaussian prior variance as one of the important CRFs parameters. Our empirical findings show that incremental learning and the Gaussian prior variance lead to more stable and robust models across iterations. Our study also demonstrates that orthographical, morphological and contextual features as a group of basic features play an important role in learning effective models across all iterations.

Global burden of disease attributable to mental and substance use disorders : findings from the Global Burden of Disease Study 2010

Background We used data from the Global Burden of Diseases, Injuries, and Risk Factors Study 2010 (GBD 2010) to estimate the burden of disease attributable to mental and substance use disorders in terms of disability-adjusted life years (DALYs), years of life lost to premature mortality (YLLs), and years lived with disability (YLDs). Methods For each of the 20 mental and substance use disorders included in GBD 2010, we systematically reviewed epidemiological data and used a Bayesian meta-regression tool, DisMod-MR, to model prevalence by age, sex, country, region, and year. We obtained disability weights from representative community surveys and an internet-based survey to calculate YLDs. We calculated premature mortality as YLLs from cause of death estimates for 1980–2010 for 20 age groups, both sexes, and 187 countries. We derived DALYs from the sum of YLDs and YLLs. We adjusted burden estimates for comorbidity and present them with 95% uncertainty intervals. Findings In 2010, mental and substance use disorders accounted for 183·9 million DALYs (95% UI 153·5 million–216·7 million), or 7·4% (6·2–8·6) of all DALYs worldwide. Such disorders accounted for 8·6 million YLLs (6·5 million–12·1 million; 0·5% [0·4–0·7] of all YLLs) and 175·3 million YLDs (144·5 million–207·8 million; 22·9% [18·6–27·2] of all YLDs). Mental and substance use disorders were the leading cause of YLDs worldwide. Depressive disorders accounted for 40·5% (31·7–49·2) of DALYs caused by mental and substance use disorders, with anxiety disorders accounting for 14·6% (11·2–18·4), illicit drug use disorders for 10·9% (8·9–13·2), alcohol use disorders for 9·6% (7·7–11·8), schizophrenia for 7·4% (5·0–9·8), bipolar disorder for 7·0% (4·4–10·3), pervasive developmental disorders for 4·2% (3·2–5·3), childhood behavioural disorders for 3·4% (2·2–4·7), and eating disorders for 1·2% (0·9–1·5). DALYs varied by age and sex, with the highest proportion of total DALYs occurring in people aged 10–29 years. The burden of mental and substance use disorders increased by 37·6% between 1990 and 2010, which for most disorders was driven by population growth and ageing. Interpretation Despite the apparently small contribution of YLLs—with deaths in people with mental disorders coded to the physical cause of death and suicide coded to the category of injuries under self-harm—our findings show the striking and growing challenge that these disorders pose for health systems in developed and developing regions. In view of the magnitude of their contribution, improvement in population health is only possible if countries make the prevention and treatment of mental and substance use disorders a public health priority.