Overview of INEX 2013

INEX investigates focused retrieval from structured documents by providing large test collections of structured documents, uniform evaluation measures, and a forum for organizations to compare their results. This paper reports on the INEX 2013 evaluation campaign, which consisted of four activities addressing three themes: searching professional and user generated data (Social Book Search track); searching structured or semantic data (Linked Data track); and focused retrieval (Snippet Retrieval and Tweet Contextualization tracks). INEX 2013 was an exciting year for INEX in which we consolidated the collaboration with (other activities in) CLEF and for the second time ran our workshop as part of the CLEF labs in order to facilitate knowledge transfer between the evaluation forums. This paper gives an overview of all the INEX 2013 tracks, their aims and task, the built test-collections, and gives an initial analysis of the results

Creation of a new evaluation benchmark for information retrieval targeting patient information needs

Searching for health advice on the web is becoming increasingly common. Because of the great importance of this activity for patients and clinicians and the effect that incorrect information may have on health outcomes, it is critical to present relevant and valuable information to a searcher. Previous evaluation campaigns on health information retrieval (IR) have provided benchmarks that have been widely used to improve health IR and record these improvements. However, in general these benchmarks have targeted the specialised information needs of physicians and other healthcare workers. In this paper, we describe the development of a new collection for evaluation of effectiveness in IR seeking to satisfy the health information needs of patients. Our methodology features a novel way to create statements of patients’ information needs using realistic short queries associated with patient discharge summaries, which provide details of patient disorders. We adopt a scenario where the patient then creates a query to seek information relating to these disorders. Thus, discharge summaries provide us with a means to create contextually driven search statements, since they may include details on the stage of the disease, family history etc. The collection will be used for the first time as part of the ShARe/-CLEF 2013 eHealth Evaluation Lab, which focuses on natural language processing and IR for clinical care.

Report on INEX 2013

INEX investigates focused retrieval from structured documents by providing large test collections of structured documents, uniform evaluation measures, and a forum for organizations to compare their results. This paper reports on the INEX 2013 evaluation campaign, which consisted of four activities addressing three themes: searching professional and user generated data (Social Book Search track); searching structured or semantic data (Linked Data track); and focused retrieval (Snippet Retrieval and Tweet Contextualization tracks). INEX 2013 was an exciting year for INEX in which we consolidated the collaboration with (other activities in) CLEF and for the second time ran our workshop as part of the CLEF labs in order to facilitate knowledge transfer between the evaluation forums. This paper gives an overview of all the INEX 2013 tracks, their aims and task, the built test-collections, and gives an initial analysis of the results.

Overview of INEX 2014

INEX investigates focused retrieval from structured documents by providing large test collections of structured documents, uniform evaluation measures, and a forum for organizations to compare their results. This paper reports on the INEX 2014 evaluation campaign, which consisted of three tracks: The Interactive Social Book Search Track investigated user information seeking behavior when interacting with various sources of information, for realistic task scenarios, and how the user interface impacts search and the search experience. The Social Book Search Track investigated the relative value of authoritative metadata and user-generated content for search and recommendation using a test collection with data from Amazon and LibraryThing, including user profiles and personal catalogues. The Tweet Contextualization Track investigated tweet contextualization, helping a user to understand a tweet by providing him with a short background summary generated from relevant Wikipedia passages aggregated into a coherent summary. INEX 2014 was an exciting year for INEX in which we for the third time ran our workshop as part of the CLEF labs. This paper gives an overview of all the INEX 2014 tracks, their aims and task, the built test-collections, the participants, and gives an initial analysis of the results.

Report on INEX 2012

INEX investigates focused retrieval from structured documents by providing large test collections of structured documents, uniform evaluation measures, and a forum for organizations to compare their results. This paper reports on the INEX'12 evaluation campaign, which consisted of a five tracks: Linked Data, Relevance Feedback, Snippet Retrieval, Social Book Search, and Tweet Contextualization. INEX'12 was an exciting year for INEX in which we joined forces with CLEF and for the first time ran our workshop as part of the CLEF labs in order to facilitate knowledge transfer between the evaluation forums.

Fine-grained plant classification using convolutional neural networks for feature extraction

We present an overview of the QUT plant classification system submitted to LifeCLEF 2014. This system uses generic features extracted from a convolutional neural network previously used to perform general object classification. We examine the effectiveness of these features to perform plant classification when used in combination with an extremely randomised forest. Using this system, with minimal tuning, we obtained relatively good results with a score of 0:249 on the test set of LifeCLEF 2014.

Active learning: A step towards automating medical concept extraction

Objective This paper presents an automatic active learning-based system for the extraction of medical concepts from clinical free-text reports. Specifically, (1) the contribution of active learning in reducing the annotation effort, and (2) the robustness of incremental active learning framework across different selection criteria and datasets is determined. Materials and methods The comparative performance of an active learning framework and a fully supervised approach were investigated to study how active learning reduces the annotation effort while achieving the same effectiveness as a supervised approach. Conditional Random Fields as the supervised method, and least confidence and information density as two selection criteria for active learning framework were used. The effect of incremental learning vs. standard learning on the robustness of the models within the active learning framework with different selection criteria was also investigated. Two clinical datasets were used for evaluation: the i2b2/VA 2010 NLP challenge and the ShARe/CLEF 2013 eHealth Evaluation Lab. Results The annotation effort saved by active learning to achieve the same effectiveness as supervised learning is up to 77%, 57%, and 46% of the total number of sequences, tokens, and concepts, respectively. Compared to the Random sampling baseline, the saving is at least doubled. Discussion Incremental active learning guarantees robustness across all selection criteria and datasets. The reduction of annotation effort is always above random sampling and longest sequence baselines. Conclusion Incremental active learning is a promising approach for building effective and robust medical concept extraction models, while significantly reducing the burden of manual annotation.

Prólogo

Homenaje a Georges Laplace, realizado en Vitoria-Gasteiz el 13,14 y 15 de noviembre de 2012. Edición a cargo de Aitor Calvo, Aitor Sánchez, Maite García-Rojas y Mónica Alonso-Eguíluz.

Francesc Gusi i Jener (1942-2012): in memoriam

Homenaje a Georges Laplace, realizado en Vitoria-Gasteiz el 13,14 y 15 de noviembre de 2012. Edición a cargo de Aitor Calvo, Aitor Sánchez, Maite García-Rojas y Mónica Alonso-Eguíluz.

Recorrido y reflexiones en torno al pensamiento analítico de Georges Laplace: movimiento, interdependencia y arquetipos en la construcción de una arqueología científica

Homenaje a Georges Laplace, realizado en Vitoria-Gasteiz el 13,14 y 15 de noviembre de 2012. Edición a cargo de Aitor Calvo, Aitor Sánchez, Maite García-Rojas y Mónica Alonso-Eguíluz.

León y caballo: Georges lLaplace. El yin y el yang de un prehistoriador inolvidable

Homenaje a Georges Laplace, realizado en Vitoria-Gasteiz el 13,14 y 15 de noviembre de 2012. Edición a cargo de Aitor Calvo, Aitor Sánchez, Maite García-Rojas y Mónica Alonso-Eguíluz.

Historia de una relación por venir: caracteres analíticos para el análisis funcional

Homenaje a Georges Laplace, realizado en Vitoria-Gasteiz el 13, 14 y 15 de noviembre de 2012. Edición a cargo de Aitor Calvo, Aitor Sánchez, Maite García-Rojas y Mónica Alonso-Eguíluz.

Arqueozoología analítica. Otro ejemplo práctico derivado de la obra de Georges Laplace

Homenaje a Georges Laplace, realizado en Vitoria-Gasteiz el 13,14 y 15 de noviembre de 2012. Edición a cargo de Aitor Calvo, Aitor Sánchez, Maite García-Rojas y Mónica Alonso-Eguíluz.

Tipología y tecnología. Una dependencia recíproca

Homenaje a Georges Laplace, realizado en Vitoria-Gasteiz el 13, 14 y 15 de noviembre de 2012. Edición a cargo de Aitor Calvo, Aitor Sánchez, Maite García-Rojas y Mónica Alonso-Eguíluz