Creation of a new evaluation benchmark for information retrieval targeting patient information needs

Searching for health advice on the web is becoming increasingly common. Because of the great importance of this activity for patients and clinicians and the effect that incorrect information may have on health outcomes, it is critical to present relevant and valuable information to a searcher. Previous evaluation campaigns on health information retrieval (IR) have provided benchmarks that have been widely used to improve health IR and record these improvements. However, in general these benchmarks have targeted the specialised information needs of physicians and other healthcare workers. In this paper, we describe the development of a new collection for evaluation of effectiveness in IR seeking to satisfy the health information needs of patients. Our methodology features a novel way to create statements of patients’ information needs using realistic short queries associated with patient discharge summaries, which provide details of patient disorders. We adopt a scenario where the patient then creates a query to seek information relating to these disorders. Thus, discharge summaries provide us with a means to create contextually driven search statements, since they may include details on the stage of the disease, family history etc. The collection will be used for the first time as part of the ShARe/-CLEF 2013 eHealth Evaluation Lab, which focuses on natural language processing and IR for clinical care.

Overview of the ShARe/CLEF eHealth Evaluation Lab 2013

Discharge summaries and other free-text reports in healthcare transfer information between working shifts and geographic locations. Patients are likely to have difficulties in understanding their content, because of their medical jargon, non-standard abbreviations,and ward-specific idioms. This paper reports on an evaluation lab with an aim to support the continuum of care by developing methods and resources that make clinical reports in English easier to understand for patients, and which helps them in finding information related to their condition.

ShARe/CLEF eHealth Evaluation Lab 2013, Task 3: Information retrieval to address patients' questions when reading clinical reports

This paper presents the results of task 3 of the ShARe/CLEF eHealth Evaluation Lab 2013. This evaluation lab focuses on improving access to medical information on the web. The task objective was to investigate the effect of using additional information such as the discharge summaries and external resources such as medical ontologies on the IR effectiveness. The participants were allowed to submit up to seven runs, one mandatory run using no additional information or external resources, and three each using or not using discharge summaries.

Rodent Scope: A User-Configurable Digital Wireless Telemetry System for Freely Behaving Animals

This paper describes the design and implementation of a wireless neural telemetry system that enables new experimental paradigms, such as neural recordings during rodent navigation in large outdoor environments. RoSco, short for Rodent Scope, is a small lightweight user-configurable module suitable for digital wireless recording from freely behaving small animals. Due to the digital transmission technology, RoSco has advantages over most other wireless modules of noise immunity and online user-configurable settings. RoSco digitally transmits entire neural waveforms for 14 of 16 channels at 20 kHz with 8-bit encoding which are streamed to the PC as standard USB audio packets. Up to 31 RoSco wireless modules can coexist in the same environment on non-overlapping independent channels. The design has spatial diversity reception via two antennas, which makes wireless communication resilient to fading and obstacles. In comparison with most existing wireless systems, this system has online user-selectable independent gain control of each channel in 8 factors from 500 to 32,000 times, two selectable ground references from a subset of channels, selectable channel grounding to disable noisy electrodes, and selectable bandwidth suitable for action potentials (300 Hz–3 kHz) and low frequency field potentials (4 Hz–3 kHz). Indoor and outdoor recordings taken from freely behaving rodents are shown to be comparable to a commercial wired system in sorting for neural populations. The module has low input referred noise, battery life of 1.5 hours and transmission losses of 0.1% up to a range of 10 m.

Evaluation and social impact measurement amongst small to medium social enterprises : process, purpose and value

In recent years, the imperative to communicate organisational impacts to a variety of stakeholders has gained increasing importance within all sectors. Despite growing external demands for evaluation and social impact measurement, there has been limited critically informed analysis about the presumed importance of these activities to organisational success and the practical challenges faced by organisations in undertaking such assessment. In this paper, we present the findings from an action research study of five Australian small to medium social enterprises’ practices and use of evaluation and social impact analysis. Our findings have implications for social enterprise operators, policy makers and social investors regarding when, why and at what level these activities contribute to organisational performance and the fulfilment of mission.

Damage evaluation based on a wave energy flow map using multiple PZT sensors

A new wave energy flow (WEF) map concept was proposed in this work. Based on it, an improved technique incorporating the laser scanning method and Betti’s reciprocal theorem was developed to evaluate the shape and size of damage as well as to realize visualization of wave propagation. In this technique, a simple signal processing algorithm was proposed to construct the WEF map when waves propagate through an inspection region, and multiple lead zirconate titanate (PZT) sensors were employed to improve inspection reliability. Various damages in aluminum and carbon fiber reinforced plastic laminated plates were experimentally and numerically evaluated to validate this technique. The results show that it can effectively evaluate the shape and size of damage from wave field variations around the damage in the WEF map.

Media innovations, user innovations, societal innovations

As the Journal of Media Innovations comes into existence, this article reflects on the first and most obvious question: just what do we mean by “media innovations”? Drawing on the examples of a range of recent innovations in media technologies and practices, initiated by a variety of media audiences, users, professionals, and providers, it explores the interplay between the different drivers of innovation and the effects of such innovation on the complex frameworks of contemporary society and the media ecology which supports it. In doing so, this article makes a number of key observations: first, it notes that media innovation is an innovation in media practices at least as much as in media technologies, and that changes to the practices of media both reflect and promote societal changes as well – media innovations are never just media technology innovations. Second, it shows that the continuing mediatisation of society, and the shift towards a more widespread participation of ordinary users as active content creators and media innovators, make it all the more important to investigate in detail these interlinked, incremental, everyday processes of media and societal change – media innovations are almost always also user innovations. Finally, it suggests that a full understanding of these processes as they unfold across diverse interleaved media spaces and complex societal structures necessarily requires a holistic perspective on media innovations, which considers the contemporary media ecology as a crucial constitutive element of societal structures and seeks to trace the repercussions of innovations across both media and society – media innovations are inextricably interlinked with societal innovations (even if, at times, they may not be considered to be improvements to the status quo).

Using performance-based evaluation to close the loop between biological and robotic navigation

In this paper we describe the benefits of a performance-based approach to modeling biological systems for use in robotics. Specifically, we describe the RatSLAM system, a computational model of the navigation processes thought to drive navigation in a part of the rodent brain called the hippocampus. Unlike typical computational modeling approaches, which focus on biological fidelity, RatSLAM’s development cycle has been driven primarily by performance evaluation on robots navigating in a wide variety of challenging, real world environments. We briefly describe three seminal results, two in robotics and one in biology. In addition, we present current research on brain-inspired learning algorithms with the aim of enabling a robot to autonomously learn how best to use its sensor suite to navigate, without requiring any specific knowledge of the robot, sensor types or environment characteristics. Our aim is to drive discussion on the merits of practical, performance-focused implementations of biological models in robotics.

Does counting still count? Revisiting the security of counting based user authentication protocols against statistical attacks

At NDSS 2012, Yan et al. analyzed the security of several challenge-response type user authentication protocols against passive observers, and proposed a generic counting based statistical attack to recover the secret of some counting based protocols given a number of observed authentication sessions. Roughly speaking, the attack is based on the fact that secret (pass) objects appear in challenges with a different probability from non-secret (decoy) objects when the responses are taken into account. Although they mentioned that a protocol susceptible to this attack should minimize this difference, they did not give details as to how this can be achieved barring a few suggestions. In this paper, we attempt to fill this gap by generalizing the attack with a much more comprehensive theoretical analysis. Our treatment is more quantitative which enables us to describe a method to theoretically estimate a lower bound on the number of sessions a protocol can be safely used against the attack. Our results include 1) two proposed fixes to make counting protocols practically safe against the attack at the cost of usability, 2) the observation that the attack can be used on non-counting based protocols too as long as challenge generation is contrived, 3) and two main design principles for user authentication protocols which can be considered as extensions of the principles from Yan et al. This detailed theoretical treatment can be used as a guideline during the design of counting based protocols to determine their susceptibility to this attack. The Foxtail protocol, one of the protocols analyzed by Yan et al., is used as a representative to illustrate our theoretical and experimental results.

Security evaluation of Rakaposhi Stream Cipher

Rakaposhi is a synchronous stream cipher, which uses three main components: a non-linear feedback shift register (NLFSR), a dynamic linear feedback shift register (DLFSR) and a non-linear filtering function (NLF). NLFSR consists of 128 bits and is initialised by the secret key K. DLFSR holds 192 bits and is initialised by an initial vector (IV). NLF takes 8-bit inputs and returns a single output bit. The work identifies weaknesses and properties of the cipher. The main observation is that the initialisation procedure has the so-called sliding property. The property can be used to launch distinguishing and key recovery attacks. The distinguisher needs four observations of the related (K,IV) pairs. The key recovery algorithm allows to discover the secret key K after observing 29 pairs of (K,IV). Based on the proposed related-key attack, the number of related (K,IV) pairs is 2(128 + 192)/4 pairs. Further the cipher is studied when the registers enter short cycles. When NLFSR is set to all ones, then the cipher degenerates to a linear feedback shift register with a non-linear filter. Consequently, the initial state (and Secret Key and IV) can be recovered with complexity 263.87. If DLFSR is set to all zeros, then NLF reduces to a low non-linearity filter function. As the result, the cipher is insecure allowing the adversary to distinguish it from a random cipher after 217 observations of keystream bits. There is also the key recovery algorithm that allows to find the secret key with complexity 2 54.

Evaluation of the Mater Mothers' Hospital refugee antenatal clinic

This report describes the evaluation of the Refugee Antenatal Clinic (Mater Mothers' Hospital, Brisbane) which was established in November 2008

An evaluation of the culture performance of two Macrobrachium species in Vanuatu: the exotic M. rosenbergii and the indigenous M. lar

Fisheries and aquaculture are important for food security, income generation and are critical to long term sustainability of many countries. Freshwater prawns have been harvested in the streams and creeks in Vanuatu, however due to over-exploitation catches have declined in recent years. To satisfy high demand for this product, Vanuatu government intends to establish economically viable small-scale aquaculture industries. The current project showed that wild Macrobrachium lar in Vanuatu constitute a single population for management purposes and that M. rosenbergii grows much faster than M. lar in simple pond grow-out systems, hence is a better species for culture in Vanuatu.

Critical reflections on the use of participatory methodologies to build evaluation capacities in international development organisations

There has been a greater focus on strengthening evaluation capacity building (ECB) within development organisations in recent years. This can be attributed in part to the growing appreciation of the value of participatory and collaborative forms of evaluation. Evaluation is increasingly seen as an ongoing learning process and an important means of strengthening capacity and improving organisational performance (Horton et al., 2003:7). While there are many benefits of using participatory methodologies in ECB projects, our experiences and a review of the literature in this area highlight the many challenges, issues and contradictions that can affect the success of such ECB efforts. We discuss these issues, drawing on our learnings from the ongoing participatory action research (PAR) project 'Assessing Communication for Social Change’ (AC4SC). This four year project, which began in 2007, is a collaboration between communication and development academics and evaluation specialists from two Australian universities and communication for development practitioners and monitoring and evaluation (M&E) staff in the NGO Equal Access Nepal (EAN). The aim is to develop, implement, and evaluate a participatory methodology for assessing the social change impacts of community radio programs produced by EAN. It builds on previous projects that used ethnographic action research (EAR) methodology (Tacchi et al., 2007).

Critical reflections on the use of participatory methodologies to build evaluation capacities in international development organisations

"In this chapter the authors present a critique of Participatory Evaluation as worked in development projects, in this case, in Nepal. The article works between established claims that Participatory Evaluation builds capacity at programmatic and organisational levels, and the specific experiences of these claims in the authors’ current work. They highlight the need to address key difficulties such as high turn-over of staff and resulting loss of capacity to engage in Participatory Evaluation, and the difficulty of communication between academic as compared with local practical wisdoms. A key issue is the challenge of addressing the inevitable issues of power inequities that such approaches encounter. While Participatory Evaluation has been around for some time, it has only enjoyed more widespread recognition of its value in comparatively recent times, with its uptake in international development environments. To this extent, the practice is still in its early stages of development, and Jo, June and Michael’s work contributes to strengthening and more comprehensively understanding it. With regard to the meta-theme of this publication, this chapter is an example of how context not only influences the methodology to be used and the praxis of how it is to be used, but contributes to early explication of the core nature of an emerging methodology."

Blast response and safety evaluation of a composite column for use as key element in structural systems

This paper presents the blast response, damage mechanism and evaluation of residual load capacity of a concrete–steel composite (CSC) column using dynamic computer simulation techniques. This study is an integral part of a comprehensive research program which investigated the vulnerability of structural framing systems to catastrophic and progressive collapse under blast loading and is intended to provide design information on blast mitigation and safety evaluation of load bearing vulnerable columns that are key elements in a building. The performance of the CSC column is compared with that of a reinforced concrete (RC) column with the same dimensions and steel ratio. Results demonstrate the superior performance of the CSC column, compared to the RC column in terms of residual load carrying capacity, and its potential for use as a key element in structural systems. The procedure and results presented herein can be used in the design and safety evaluation of key elements of multi-storey buildings for mitigating the impact of blast loads.