Integrating understandability in the evaluation of consumer health search engines

In this paper we propose a method that integrates the no- tion of understandability, as a factor of document relevance, into the evaluation of information retrieval systems for con- sumer health search. We consider the gain-discount evaluation framework (RBP, nDCG, ERR) and propose two understandability-based variants (uRBP) of rank biased precision, characterised by an estimation of understandability based on document readability and by different models of how readability influences user understanding of document content. The proposed uRBP measures are empirically contrasted to RBP by comparing system rankings obtained with each measure. The findings suggest that considering understandability along with topicality in the evaluation of in- formation retrieval systems lead to different claims about systems effectiveness than considering topicality alone.

Evaluation of public hospital performance in Khanh Hoa Province : Vietnam in connection with patient and staff satisfaction surveys

This research aimed to develop a framework for performance evaluation of public hospitals in Vietnam that is culturally, socially, and politically appropriate. The research included both qualitative and quantitative methods and identified and validated novel instruments to measure patient satisfaction and job satisfaction of hospital staff and to determine a set of hospital indicators that reflect the quality of hospital performance. New models for understanding the determinants of patient and staff satisfaction were developed along with a new performance indicator framework for hospital performance. These instruments will now be applied to the evaluation of hospital services in Khanh Hoa Province, permitting longer term evaluation of their effectiveness in changing system wide performance and satisfaction.

Benefit evaluation for information providing system by WebGIS in view of users' travel costs

This paper estimates the benefit of a plan for information providing system on road administration by WebGIS. The system will reduce travel costs of visitors from their business establishments to a road administration section of a city office. The authors had individual interviews with the visitors at the section of the Ichikawa City Office. Annual total sum of travel costs was estimated at 37 million yen at most. This paper also proposes formulas which expect the frequency of visits or the total sum of travel costs from the spatial distribution of the business establishments without questionnaires.

Beyond point forecasting : evaluation of alternative prediction intervals for tourist arrivals

This paper evaluates the performances of prediction intervals generated from alternative time series models, in the context of tourism forecasting. The forecasting methods considered include the autoregressive (AR) model, the AR model using the bias-corrected bootstrap, seasonal ARIMA models, innovations state space models for exponential smoothing, and Harvey’s structural time series models. We use thirteen monthly time series for the number of tourist arrivals to Hong Kong and Australia. The mean coverage rates and widths of the alternative prediction intervals are evaluated in an empirical setting. It is found that all models produce satisfactory prediction intervals, except for the autoregressive model. In particular, those based on the biascorrected bootstrap perform best in general, providing tight intervals with accurate coverage rates, especially when the forecast horizon is long.

Wellness programs at firefighter and police workplaces : a systematic review

Objectives This systematic review summarizes the literature on the health effects of wellness programs at police and firefighter workplaces. The review also considers process outcomes and economic evaluations of such programs. Methods A systematic search of articles published from January 1, 2000 through September 1, 2012 in 13 databases was conducted. Data on 7 studies from 9 articles were extracted. An assessment of the methodological quality of the studies was conducted. Results Studies showed acceptable completion rates and high satisfaction with programs, but mixed results for changes in individual outcomes. Conclusion Empirical evidence for effective health and wellness programs in police and firefighter populations is scarce. Better evaluation and documentation of such programs is needed to advance this field of research.

Performance evaluation of three-level Z-source inverters under semiconductor-failure conditions

This paper evaluates and proposes various compensation methods for three-level Z-source inverters under semiconductor-failure conditions. Unlike the fault-tolerant techniques used in traditional three-level inverters, where either an extra phase-leg or collective switching states are used, the proposed methods for three-level Z-source inverters simply reconfigure their relevant gating signals so as to ride-through the failed semiconductor conditions smoothly without any significant decrease in their ac-output quality and amplitude. These features are partly attributed to the inherent boost characteristics of a Z-source inverter, in addition to its usual voltage-buck operation. By focusing on specific types of three-level Z-source inverters, it can also be shown that, for the dual Z-source inverters, a unique feature accompanying it is its extra ability to force common-mode voltage to zero even under semiconductor-failure conditions. For verifying these described performance features, PLECS simulation and experimental testing were performed with some results captured and shown in a later section for visual confirmation.

Evaluation of resonant damping techniques for Z-source current-type inverter

For the renewable energy sources whose outputs vary continuously, a Z-source current-type inverter has been proposed as a possible buck-boost alternative for grid-interfacing. With a unique X-shaped LC network connected between its dc power source and inverter topology, Z-source current-type inverter is however expected to suffer from compounded resonant complications in addition to those associated with its second-order output filter. To improve its damping performance, this paper proposes the careful integration of Posicast or three-step compensators before the inverter pulse-width modulator for damping triggered resonant oscillations. In total, two compensators are needed for wave-shaping the inverter boost factor and modulation ratio, and they can conveniently be implemented using first-in first-out stacks and embedded timers of modern digital signal processors widely used in motion control applications. Both techniques are found to damp resonance of ac filter well, but for cases of transiting from current-buck to boost state, three-step technique is less effective due to the sudden intermediate discharging interval introduced by its non-monotonic stepping (unlike the monotonic stepping of Posicast damping). These findings have been confirmed both in simulations and experiments using an implemented laboratory prototype.

Rapid on-site evaluation of fine needle aspiration specimens by cytology scientists : a review of 3032 specimens

Objectives To determine: (1) the accuracy of cytology scientists at assessing specimen adequacy by rapid on-site evaluation (ROSE) at fine needle aspiration (FNA) cytology collections; and (2) whether thyroid FNA with ROSE has lower inadequacy rates than non-attended FNAs. Methods The ROSE of adequacy for 3032 specimens from 17 anatomical sites collected over a 20-month period was compared with the final report assessment of adequacy. ROSE was performed by 19 cytology scientists. The report profile for 1545 thyroid nodules with ROSE was compared with that for 1536 consecutive non-ROSE thyroid FNAs reported by the same cytopathologists during the study period. Results ROSE was adequate in 75% (2276/3032), inadequate in 12% (366/3032) and in 13% (390/3032) no opinion was rendered. Of the 2276 cases assessed as adequate by ROSE, 2268 (99.6%) were finally reported as adequate for assessment; eight specimens had adequacy downgraded on the final report. Fifty eight per cent of cases with a ROSE assessment of inadequate were reported as adequate (212/366), whereas 93% (363/390) with no opinion rendered were reported as adequate. The overall final report adequacy rate for the 3032 specimens was 94% (2843/3032). Confirmation of a ROSE of adequacy at reporting was uniformly high amongst the 19 scientists, ranging from 98% to 100%. The inadequacy rate for thyroid FNAs with ROSE (6%) was significantly (P < 0.0001) lower than for non-ROSE thyroid FNAs (17%). A significantly (P = 0.02) higher proportion of adequate ROSE thyroid specimens was reported with abnormalities, compared with non-ROSE thyroid collections. Conclusions Cytology scientists are highly accurate at determining specimen adequacy at ROSE for a wide range of body sites. ROSE of thyroid FNAs can significantly reduce inadequate reports.

Development and evaluation of the Ontrack Diabetes Program : an automated, web-based type 2 diabetes self-management and dysphoria intervention

This research was conducted in the area of Clinical and Health Psychology. The study involved the development and evaluation of a novel, web-based program aimed to improve Type 2 diabetes self-management and mood. The program was developed as an original technological intervention aimed to improve access to support for rural and remote communities, and is currently being trialled across Australia with a larger sample size. The researcher aims to continue research into the field of clinical psychology, and in particular is interested in working on further interventions to support those with comorbid physical and mental health conditions.

Evaluation of emerging intelligent transport systems to improve safety on level crossings : an overview

Safety at railway level crossings (RLX) is one part of a wider picture of safety within the whole transport system. Governments, the rail industry and road organisations have used a variety of countermeasures for many years to improve RLX safety. New types of interventions are required in order to reduce the number of crashes and associated social costs at railway crossings. This paper presents the results of a large research program which aimed to assess the effectiveness of emerging Intelligent Transport Systems (ITS) interventions, both on-road and in-vehicle based, to improve the safety of car drivers at RLXs in Australia. The three most promising technologies selected from the literature review and focus groups were tested in an advanced driving simulator to provide a detailed assessment of their effects on driver behaviour. The three interventions were: (i) in-vehicle visual warning using a GPS/smartphone navigation-like system, (ii) in-vehicle audio warning and; (iii) on-road intervention known as valet system (warning lights on the road surface activated as a train approaches). The effects of these technologies on 57 participants were assessed in a systematic approach focusing on the safety of the intervention, effects on the road traffic around the crossings and driver’s acceptance of the technology. Given that the ITS interventions were likely to provide a benefit by improving the driver’s awareness of the crossing status in low visibility conditions, such conditions were investigated through curves in the track before arriving at the crossing. ITS interventions were also expected to improve driver behaviour at crossings with high traffic (blocking back issue), which were also investigated at active crossings. The key findings are: (i) interventions at passive crossings are likely to provide safety benefits; (ii) the benefits of ITS interventions on driver behaviour at active crossings are limited; (iii) the trialled ITS interventions did not show any issues in terms of driver distraction, driver acceptance or traffic delays; (iv) these interventions are easy to use, do not increase driver workload substantially; (v) participants’ intention to use the technology is high and; (vi) participants saw most value in succinct messages about approaching trains as opposed to knowing the RLX locations or the imminence of a collision with a train.

Blood transfer devices for malaria rapid diagnostic tests : evaluation of accuracy, safety and ease of use

Background: Malaria rapid diagnostic tests (RDTs) are increasingly used by remote health personnel with minimal training in laboratory techniques. RDTs must, therefore, be as simple, safe and reliable as possible. Transfer of blood from the patient to the RDT is critical to safety and accuracy, and poses a significant challenge to many users. Blood transfer devices were evaluated for accuracy and precision of volume transferred, safety and ease of use, to identify the most appropriate devices for use with RDTs in routine clinical care. Methods: Five devices, a loop, straw-pipette, calibrated pipette, glass capillary tube, and a new inverted cup device, were evaluated in Nigeria, the Philippines and Uganda. The 227 participating health workers used each device to transfer blood from a simulated finger-prick site to filter paper. For each transfer, the number of attempts required to collect and deposit blood and any spilling of blood during transfer were recorded. Perceptions of ease of use and safety of each device were recorded for each participant. Blood volume transferred was calculated from the area of blood spots deposited on filter paper. Results: The overall mean volumes transferred by devices differed significantly from the target volume of 5 microliters (p < 0.001). The inverted cup (4.6 microliters) most closely approximated the target volume. The glass capillary was excluded from volume analysis as the estimation method used is not compatible with this device. The calibrated pipette accounted for the largest proportion of blood exposures (23/225, 10%); exposures ranged from 2% to 6% for the other four devices. The inverted cup was considered easiest to use in blood collection (206/ 226, 91%); the straw-pipette and calibrated pipette were rated lowest (143/225 [64%] and 135/225 [60%] respectively). Overall, the inverted cup was the most preferred device (72%, 163/227), followed by the loop (61%, 138/227). Conclusions: The performance of blood transfer devices varied in this evaluation of accuracy, blood safety, ease of use, and user preference. The inverted cup design achieved the highest overall performance, while the loop also performed well. These findings have relevance for any point-of-care diagnostics that require blood sampling.

Overview of the ShARe/CLEF eHealth Evaluation Lab 2014

This paper reports on the 2nd ShARe/CLEFeHealth evaluation lab which continues our evaluation resource building activities for the medical domain. In this lab we focus on patients' information needs as opposed to the more common campaign focus of the specialised information needs of physicians and other healthcare workers. The usage scenario of the lab is to ease patients and next-of-kins' ease in understanding eHealth information, in particular clinical reports. The 1st ShARe/CLEFeHealth evaluation lab was held in 2013. This lab consisted of three tasks. Task 1 focused on named entity recognition and normalization of disorders; Task 2 on normalization of acronyms/abbreviations; and Task 3 on information retrieval to address questions patients may have when reading clinical reports. This year's lab introduces a new challenge in Task 1 on visual-interactive search and exploration of eHealth data. Its aim is to help patients (or their next-of-kin) in readability issues related to their hospital discharge documents and related information search on the Internet. Task 2 then continues the information extraction work of the 2013 lab, specifically focusing on disorder attribute identification and normalization from clinical text. Finally, this year's Task 3 further extends the 2013 information retrieval task, by cleaning the 2013 document collection and introducing a new query generation method and multilingual queries. De-identified clinical reports used by the three tasks were from US intensive care and originated from the MIMIC II database. Other text documents for Tasks 1 and 3 were from the Internet and originated from the Khresmoi project. Task 2 annotations originated from the ShARe annotations. For Tasks 1 and 3, new annotations, queries, and relevance assessments were created. 50, 79, and 91 people registered their interest in Tasks 1, 2, and 3, respectively. 24 unique teams participated with 1, 10, and 14 teams in Tasks 1, 2 and 3, respectively. The teams were from Africa, Asia, Canada, Europe, and North America. The Task 1 submission, reviewed by 5 expert peers, related to the task evaluation category of Effective use of interaction and targeted the needs of both expert and novice users. The best system had an Accuracy of 0.868 in Task 2a, an F1-score of 0.576 in Task 2b, and Precision at 10 (P@10) of 0.756 in Task 3. The results demonstrate the substantial community interest and capabilities of these systems in making clinical reports easier to understand for patients. The organisers have made data and tools available for future research and development.

Course evaluation : reconfigurations for learning with learning management systems

The introduction of online delivery platforms such as learning management systems (LMS) in tertiary education has changed the methods and modes of curriculum delivery and communication. While course evaluation methods have also changed from paper-based in-class-administered methods to largely online-administered methods, the data collection instruments have remained unchanged. This paper reports on a small exploratory study of two tertiary-level courses. The study investigated why design of the instruments and methods to administer surveys in the courses are ineffective measures against the intrinsic characteristics of online learning. It reviewed the students' response rates of the conventional evaluations for the courses over an eight-year period. It then compared a newly developed online evaluation and the conventional methods over a two-year period. The results showed the response rates with the new evaluation method increased by more than 80% from the average of the conventional evaluations (below 30%), and the students' written feedback was more detailed and comprehensive than in the conventional evaluations. The study demonstrated the possibility that the LMS-based learning evaluation can be effective and efficient in terms of the quality of students' participation and engagement in their learning, and for an integrated pedagogical approach in an online learning environment.

Stature estimation of a contemporary Australian sub-population: An evaluation of the Trotter and Gleser method using computed tomography of the femur

Utilising computed tomography scans to allow a virtual analysis of three-dimensional reconstructions of the femur, this project confirms that the traditional 1952 Trotter and Gleser stature estimation equations are inapplicable for a contemporary Queensland population. Therefore, this study introduces modern stature estimation equations for femoral length and fragmentary femoral remains using Bayesian statistics for application in forensic anthropological casework. In addition, it was found that caution needs to be applied when comparing estimated stature to reported stature on the missing persons database due to inaccuracy in Queensland drivers' licences.