Overview of the ShARe/CLEF eHealth Evaluation Lab 2014

This paper reports on the 2nd ShARe/CLEFeHealth evaluation lab which continues our evaluation resource building activities for the medical domain. In this lab we focus on patients' information needs as opposed to the more common campaign focus of the specialised information needs of physicians and other healthcare workers. The usage scenario of the lab is to ease patients and next-of-kins' ease in understanding eHealth information, in particular clinical reports. The 1st ShARe/CLEFeHealth evaluation lab was held in 2013. This lab consisted of three tasks. Task 1 focused on named entity recognition and normalization of disorders; Task 2 on normalization of acronyms/abbreviations; and Task 3 on information retrieval to address questions patients may have when reading clinical reports. This year's lab introduces a new challenge in Task 1 on visual-interactive search and exploration of eHealth data. Its aim is to help patients (or their next-of-kin) in readability issues related to their hospital discharge documents and related information search on the Internet. Task 2 then continues the information extraction work of the 2013 lab, specifically focusing on disorder attribute identification and normalization from clinical text. Finally, this year's Task 3 further extends the 2013 information retrieval task, by cleaning the 2013 document collection and introducing a new query generation method and multilingual queries. De-identified clinical reports used by the three tasks were from US intensive care and originated from the MIMIC II database. Other text documents for Tasks 1 and 3 were from the Internet and originated from the Khresmoi project. Task 2 annotations originated from the ShARe annotations. For Tasks 1 and 3, new annotations, queries, and relevance assessments were created. 50, 79, and 91 people registered their interest in Tasks 1, 2, and 3, respectively. 24 unique teams participated with 1, 10, and 14 teams in Tasks 1, 2 and 3, respectively. The teams were from Africa, Asia, Canada, Europe, and North America. The Task 1 submission, reviewed by 5 expert peers, related to the task evaluation category of Effective use of interaction and targeted the needs of both expert and novice users. The best system had an Accuracy of 0.868 in Task 2a, an F1-score of 0.576 in Task 2b, and Precision at 10 (P@10) of 0.756 in Task 3. The results demonstrate the substantial community interest and capabilities of these systems in making clinical reports easier to understand for patients. The organisers have made data and tools available for future research and development.

Acceptability-based QoE management for user-centric mobile video delivery : a field study evaluation

Effective Quality of Experience (QoE) management for mobile video delivery – to optimize overall user experience while adapting to heterogeneous use contexts – is still a big challenge to date. This paper proposes a mobile video delivery system to emphasize the use of acceptability as the main indicator of QoE to manage the end-to-end factors in delivering mobile video services. The first contribution is a novel framework for user-centric mobile video system that is based on acceptability-based QoE (A-QoE) prediction models, which were derived from comprehensive subjective studies. The second contribution is results from a field study that evaluates the user experience of the proposed system during realistic usage circumstances, addressing the impacts of perceived video quality, loading speed, interest in content, viewing locations, network bandwidth, display devices, and different video coding approaches, including region-of-interest (ROI) enhancement and center zooming

Reframe : QUT's Evaluation Framework 2014 Communications Plan

2014 is the year for embedding the Reframe: QUT’s Evaluation Framework as core business within the University, with the following providing an outline of the Learning and Teaching Unit’s (LTU) planned activities to support this implementation. LTU is continuing its commitment to intensive support and engagement with the academic and teaching focused community within QUT. The Academic Quality and Standards team have extended the existing communication and dissemination activities into a comprehensive communication plan for 2014, with major initiatives defined within this document. It should be noted that these activities are aligned to the discussions from the 2013 Integrated Management of Feedback (IMF) Steering Group and as endorsed by the University Learning and Teaching Committee at their meeting in February, 2014.

Reducing rear-end crashes with cooperative systems

This paper presents an evaluation of the effectiveness of a cooperative Intelligent Transport System (C-ITS) to reduce rear-end crashes. Two complementary simulation techniques are used to demonstrate the benefits of the C-ITS. A traffic (VEINS) and sensor (SiVIC) simulations use realistic data related to traffic/road in Brisbane’s Pacific Motorway, driver’s reaction time and injury severity to evaluate benefits. The results of our simulations show that C-ITS could reduce rear-end crash risk by providing several seconds of additional warning to drivers.

Assessment of robust autonomy for unmanned systems : progress and challenges

The growing number of potential applications of Unmanned Aircraft Systems (UAS) in civilian operations and national security is putting pressure of National Airworthiness Authorities to provide a path for certification and allow UAS integration into the national airspace. The success of this integration depends not only on developments in improved UAS reliability and safety, but also on regulations for certification, and methodologies for operational performance and safety assessment. This paper focuses on the latter and describes progress in relation to a previously proposed framework for evaluating robust autonomy of UAS. The paper draws parallels between the proposed evaluation framework and the evaluation of pilots during the licensing process. It discusses how the data from the proposed evaluation can be used as an aid for decision making in certification and UAS designs. Finally, it discusses challenges associated with the evaluation.

An interpretative structural modeling based network reconfiguration strategy for power systems

The network reconfiguration is an important stage of restoring a power system after a complete blackout or a local outage. Reasonable planning of the network reconfiguration procedure is essential for rapidly restoring the power system concerned. An approach for evaluating the importance of a line is first proposed based on the line contraction concept. Then, the interpretative structural modeling (ISM) is employed to analyze the relationship among the factors having impacts on the network reconfiguration. The security and speediness of restoring generating units are considered with priority, and a method is next proposed to select the generating unit to be restored by maximizing the restoration benefit with both the generation capacity of the restored generating unit and the importance of the line in the restoration path considered. Both the start-up sequence of generating units and the related restoration paths are optimized together in the proposed method, and in this way the shortcomings of separately solving these two issues in the existing methods are avoided. Finally, the New England 10-unit 39-bus power system and the Guangdong power system in South China are employed to demonstrate the basic features of the proposed method.

Implementation and evaluation of open source unified communications for SMBs

Unified Communication (UC) is the integration of two or more real time communication systems into one platform. Integrating core communication systems into one overall enterprise level system delivers more than just cost saving. These real-time interactive communication services and applications over Internet Protocol (IP) have become critical in boosting employee accessibility and efficiency, improving customer support and fostering business agility. However, some small and medium-sized businesses (SMBs) are far from implementing this solution due to the high cost of initial deployment and ongoing support. In this paper, we will discuss and demonstrate an open source UC solution, viz. “Asterisk” for use by SMBs, and report on some performance tests using SIPp. The contribution from this research is the provision of technical advice to SMBs in deploying UC, which is manageable in terms of cost, ease of deployment and support.

An evaluation of the neocognitron

We describe a sequence of experiments investigating the strengths and limitations of Fukushima's neocognitron as a handwritten digit classifier. Using the results of these experiments as a foundation, we propose and evaluate improvements to Fukushima's original network in an effort to obtain higher recognition performance. The neocognitron's performance is shown to be strongly dependent on the choice of selectivity parameters and we present two methods to adjust these variables. Performance of the network under the more effective of the two new selectivity adjustment techniques suggests that the network fails to exploit the features that distinguish different classes of input data. To avoid this shortcoming, the network's final layer cells were replaced by a nonlinear classifier (a multilayer perceptron) to create a hybrid architecture. Tests of Fukushima's original system and the novel systems proposed in this paper suggest that it may be difficult for the neocognitron to achieve the performance of existing digit classifiers due to its reliance upon the supervisor's choice of selectivity parameters and training data. These findings pertain to Fukushima's implementation of the system and should not be seen as diminishing the practical significance of the concept of hierarchical feature extraction embodied in the neocognitron. © 1997 IEEE.

Evaluation of channel congestion in dedicated short-range communication networks

This thesis presents a novel idea for an adaptive prioritized cross-layer design (APCLD) control algorithm to achieve comprehensive channel congestion control for vehicular safety communication based on DSRC technology. An appropriate evaluation metric and two control parameters have been established. Simulation studies have evaluated the DSRC network performance in different traffic scenario and under different channel conditions. The APCLD algorithm is derived from the results of the simulation analysis.

Visual control for automated aircraft collision avoidance systems

This thesis presents a new vision-based decision and control strategy for automated aircraft collision avoidance that can be realistically applied to the See and Avoid problem. The effectiveness of the control strategy positions the research as a major contribution toward realising the simultaneous operation of manned and unmanned aircraft within civilian airspace. Key developments include novel classical and visual predictive control frameworks, and a performance evaluation technique aligned with existing aviation practise and applicable to autonomous systems. The overall approach is demonstrated through experimental results on a small multirotor unmanned aircraft, and through high fidelity probabilistic simulation studies.

Course evidence matrix : a collaborative evaluation using multiple lines of evidence

Welcome to the Evaluation of course matrix. This matrix is designed for highly qualified discipline experts to evaluate their course, major or unit in a systemic manner. The primary purpose of the Evaluation of course matrix is to provide a tool that a group of academic staff at universities can collaboratively review the assessment within a course, major or unit annually. The annual review will result in you being ready for an external curricula review at any point in time. This tool is designed for use in a workshop format with one, two or more academic staff, and will lead to an action plan for implementation. I hope you find this tool useful in your assessment review.

Performance evaluation of cloud-based parallel computing

As computational models in fields such as medicine and engineering get more refined, resource requirements are increased. In a first instance, these needs have been satisfied using parallel computing and HPC clusters. However, such systems are often costly and lack flexibility. HPC users are therefore tempted to move to elastic HPC using cloud services. One difficulty in making this transition is that HPC and cloud systems are different, and performance may vary. The purpose of this study is to evaluate cloud services as a means to minimise both cost and computation time for large-scale simulations, and to identify which system properties have the most significant impact on performance. Our simulation results show that, while the performance of Virtual CPU (VCPU) is satisfactory, network throughput may lead to difficulties.