Qualitative evaluation of KA24 (Knowledge Access 24)

Durbin, J. & Urquhart, C. (2003). Qualitative evaluation of KA24 (Knowledge Access 24). Aberystwyth: Department of Information Studies, University of Wales Aberystwyth. Sponsorship: Knowledge Access 24 (NHS)

Understanding student information behavior in relation to electronic information services: lessons from longitudinal monitoring and evaluation Part 1

Rowley, J.& Urquhart, C. (2007). Understanding student information behavior in relation to electronic information services: lessons from longitudinal monitoring and evaluation Part 1. Journal of the American Society for Information Science and Technology, 58(8), 1162-1174. Sponsorship: JISC

Understanding student information behavior in relation to electronic information services: lessons from longitudinal monitoring and evaluation Part 2

Urquhart, C. & Rowley, J. (2007). Understanding student information behavior in relation to electronic information services: lessons from longitudinal monitoring and evaluation Part 2. Journal of the American Society for Information Science and Technology, 58(8), 1188-1197. Sponsorship: JISC

Semi-Supervised OWA Aggregation for Link-Based Similarity Evaluation and Alias Detection

T.Boongoen and Q. Shen. Semi-Supervised OWA Aggregation for Link-Based Similarity Evaluation and Alias Detection. Proceedings of the 18th International Conference on Fuzzy Systems (FUZZ-IEEE'09), pp. 288-293, 2009. Sponsorship: EPSRC

Evaluation of the quality of the management of cancer of the corpus uteri-Selection of relevant quality indicators and implementation in Belgium

Objective Describe the methodology and selection of quality indicators (QI) to be implemented in the EFFECT (EFFectiveness of Endometrial Cancer Treatment) project. EFFECT aims to monitor the variability in Quality of Care (QoC) of uterine cancer in Belgium, to compare the effectiveness of different treatment strategies to improve the QoC and to check the internal validity of the QI to validate the impact of process indicators on outcome. Methods A QI list was retrieved from literature, recent guidelines and QI databases. The Belgian Healthcare Knowledge Center methodology was used for the selection process and involved an expert's panel rating the QI on 4 criteria. The resulting scores and further discussion resulted in a final QI list. An online EFFECT module was developed by the Belgian Cancer Registry including the list of variables required for measuring the QI. Three test phases were performed to evaluate the relevance, feasibility and understanding of the variables and to test the compatibility of the dataset. Results 138 QI were considered for further discussion and 82 QI were eligible for rating. Based on the rating scores and consensus among the expert's panel, 41 QI were considered measurable and relevant. Testing of the data collection enabled optimization of the content and the user-friendliness of the dataset and online module. Conclusions This first Belgian initiative for monitoring the QoC of uterine cancer indicates that the previously used QI selection methodology is reproducible for uterine cancer. The QI list could be applied by other research groups for comparison. © 2013 Elsevier Inc.

Making sense of game based user data: learning analytics in applied games

Digital learning games are useful educational tools with high motivational potential. With the application of games for instruction there comes the need of acknowledging learning game experiences also in the context of educational assessment. Learning analytics provides new opportunities for supporting assessment in and of educational games. We give an overview of current learning analytics methods in this field and reflect on existing challenges. An approach of providing reusable software assets for interaction assessment and evaluation in games is presented. This is part of a broader initiative of making available advanced methodologies and tools for supporting applied game development.

The evaluation of time series: their scientific value and contribution to policy needs. Occasional Publication of the Marine Biological Association 22

In 2000 a Review of Current Marine Observations in relation to present and future needs was undertaken by the Inter-Agency Committee for Marine Science and Technology (IACMST). The Marine Environmental Change Network (MECN) was initiated in 2002 as a direct response to the recommendations of the report. A key part of the current phase of the MECN is to ensure that information from the network is provided to policy makers and other end-users to enable them to produce more accurate assessments of ecosystem state and gain a clearer understanding of factors influencing change in marine ecosystems. The MECN holds workshops on an annual basis, bringing together partners maintaining time-series and long-term datasets as well as end-users interested in outputs from the network. It was decided that the first workshop of the MECN continuation phase should consist of an evaluation of the time series and data sets maintained by partners in the MECN with regard to their ‘fit for purpose’ for answering key science questions and informing policy development. This report is based on the outcomes of the workshop. Section one of the report contains a brief introduction to monitoring, time series and long-term datasets. The various terms are defined and the need for MECN type data to complement compliance monitoring programmes is discussed. Outlines are also given of initiatives such as the United Kingdom Marine Monitoring and Assessment Strategy (UKMMAS) and Oceans 2025. Section two contains detailed information for each of the MECN time series / long-term datasets including information on scientific outputs and current objectives. This information is mainly based on the presentations given at the workshop and therefore follows a format whereby the following headings are addressed: Origin of time series including original objectives; current objectives; policy relevance; products (advice, publications, science and society). Section three consists of comments made by the review panel concerning all the time series and the network. Needs or issues highlighted by the panel with regard to the future of long-term datasets and time-series in the UK are shown along with advice and potential solutions where offered. The recommendations are divided into 4 categories; ‘The MECN and end-user requirements’; ‘Procedures & protocols’; ‘Securing data series’ and ‘Future developments’. Ever since marine environmental protection issues really came to the fore in the 1960s, it has been recognised that there is a requirement for a suitable evidence base on environmental change in order to support policy and management for UK waters. Section four gives a brief summary of the development of marine policy in the UK along with comments on the availability and necessity of long-term marine observations for the implementation of this policy. Policy relating to three main areas is discussed; Marine Conservation (protecting biodiversity and marine ecosystems); Marine Pollution and Fisheries. The conclusion of this section is that there has always been a specific requirement for information on long-term change in marine ecosystems around the UK in order to address concerns over pollution, fishing and general conservation. It is now imperative that this need is addressed in order for the UK to be able to fulfil its policy commitments and manage marine ecosystems in the light of climate change and other factors.

Overcoming health inequalities: A participative evaluation framework fit for the task

Aims: Healthcare providers are confronted with the claim that the distribution of health and healthcare provision is inherently unfair. There is also a growing awareness that the tools and methodologies applied in tackling health inequalities require further development. Evaluations as well as interventions usually focus on population-based indicators, but do not always provide guidance for frontline service evaluation and delivery. That is why the evaluation framework presented here focuses on facilitating local service development, service provider and user involvement, and the adequate representation of different population groups. Methods: A participative evaluation framework was constructed by drawing on six common success characteristics extrapolated from the published literature and policies on health inequalities. This framework was then applied to an intervention addressing women’s psychosocial health needs in order to demonstrate its utility in practice. Results: The framework provides healthcare professionals with an evidence-based tool for evaluating projects or programmes targeting health inequalities in ways that are responsive to local contexts and stakeholders. Conclusion: This participative evaluation framework supports the identification of meaningful psychosocial and contextual indicators for assessing the diverse health and social needs of service users. It uses multi-dimensional indicators to assess health and social care needs, to inform local service development, and to facilitate the exchange of knowledge between researchers, service providers, and service users. The inherent responsiveness enables rigorous yet flexible action on local health inequalities.

A virtual surgery in general practice: evaluation of a novel undergraduate virtual patient learning package

Background: A suite of 10 online virtual patients developed using the IVIMEDS ‘Riverside’ authoring tool has been introduced into our undergraduate general practice clerkship. These cases provide a multimedia-rich experience to students. Their interactive nature promotes the development of clinical reasoning skills such as discriminating key clinical features, integrating information from a variety of sources and forming diagnoses and management plans.

Aims: To evaluate the usefulness and usability of a set of online virtual patients in an undergraduate general practice clerkship.
Method: Online questionnaire completed by students after their general practice placement incorporating the System Usability Scale questionnaire.

Results: There was a 57% response rate. Ninety-five per cent of students agreed that the online package was a useful learning tool and ranked virtual patients third out of six learning modalities. Questions and answers and the use of images and videos were all rated highly by students as useful learning methods. The package was perceived to have a high level of usability among respondents.

Conclusion: Feedback from students suggest that this implementation of virtual patients, set in primary care, is user friendly and rated as a valuable adjunct to their learning. The cost of production of such learning resources demands close attention to design.

A techno-economic evaluation of changing from conventional vehicles to range extended electric vehicles

Transportation accounts for 22% of greenhouse gas emissions in the UK, and increases to 25% in Northern Ireland. Surface transport carbon dioxide emissions, consisting of road and rail, are dominated by cars. Demand for mobility is rising rapidly and vehicle numbers are expected to more than double by 2050. Car manufacturers are working towards reducing their carbon footprint through improving fuel efficiency and controlling exhaust emissions. Fuel efficiency is now a key consideration of consumers purchasing a new vehicle. While measures have been taken to help to reduce pollutants, in the future, alternative technologies will have to be used in the transportation industry to achieve sustainability. There are currently many alternatives to the market leader, the internal combustion engine. These alternatives include hydrogen fuel cell vehicles and electric vehicles, a term which is widely used to cover battery electric vehicles, plug-in hybrid electric vehicles and extended-range electric vehicles. This study draws direct comparisons measuring the differing performance in terms of fuel consumption, carbon emissions and range of a typical family saloon car using different fuel types. These comparisons will then be analysed to see what effect switching from a conventionally fuelled vehicle to a range extended electric vehicle would have not only on the end user, but also the UK government.

‘Making it real’: evaluating the impact of service user and carer involvement in social work education

Service user and carer involvement (SUCI) in social work education in England is required by the profession’s regulator, the Health and Care Professions Council. However, a recent study of 83 HEIs in England reported that despite considerable progress in SUCI, there is no evidence that the learning derived from it is being transferred to social work practice. In this article we describe a study that examines the question: ‘What impact does SUCI have on the skills, knowledge and values of student social workers at the point of qualification and beyond?’ Students at universities in England and Northern Ireland completed online questionnaires and participated in focus groups, spanning a period immediately pre-qualification and between six to nine months post-qualification. From our findings, we identify four categories that influence the impact of service user involvement on students’ learning: student factors; service user and carer factors; programme factors; and practice factors; each comprises of a number of sub-categories. We propose that the model developed can be used by social work educators, service user and carer contributors and practitioners to maximise the impact of SUCI. We argue that our findings also have implications for employment-based learning routes and post-qualifying education.

An Evaluation of the Dual Diagnosis (substance use and mental health )Strategy in the South Eastern Trust Workers and Service Users Perceptions of Dual Diagnosis Issues

In the UK it is estimated that over 33% of psychiatric patients with enduring mental illness have a substance misuse problem, whilst over 50 % of clients currently accessing drug and alcohol services have a mental health problem. Between 2003 and 2013 in Northern Ireland, there were 741 recorded suicides by patients who were in contact with mental health services. Of this number, 68% (n=501) had a history of either alcohol or drug misuse or both, resulting in an average of 46 patient suicides per year associated with dual diagnosis (University of Manchester 2015).
The current evaluation examined staff attitudes towards working with dual diagnosis (co-existing difficulties) issues, staff confidence in working with clients with dual diagnosis, workers’ perceptions of the South Eastern dual diagnosis strategy and service user perspectives of dual diagnosis service provision.
The purpose of the evaluation was to provide evidence regarding the following in accordance with the current dual diagnosis strategy;
Staff understanding of the concept of dual diagnosis,
Staff attitudes towards working with dual diagnosis,
Staff confidence in working with individuals, who present with dual diagnosis,
Service users’ perspectives of SE Trust provision for dual diagnosis.
Staff views on the South Eastern Trust Dual Diagnosis Strategy.

Evaluation of the human factor in the scheduling of smart appliances in smart grids

Recently there has been an increase of interest in implementing a new set of home appliances, known as Smart Appliances that integrate Information Technologies, the Internet of Things and the ability of communicating with other devices. While Smart Appliances are characterized as an important milestone on the path to the Smart Grid, by being able to automatically schedule their loads according to a tariff or reflecting the power that is generated using renewable sources, there is not a clear understanding on the impact that the behavior of such devices will have in the comfort levels of users, when they shift their working periods to earlier, or later than, a preset time. Given these considerations, in this work we analyse the results of an assessment survey carried out to a group of Home Appliance users regarding their habits when dealing with these machines and the subjective impact in quality caused by either finishing its programs before or after the time limit set by the user. The results of this work are expected to be used as input for the evaluation of load scheduling algorithms running in energy management systems. © 2014 Springer International Publishing.

Hybrid authentication systems: towards robust user authentication on wireless and mobile devices using biometric technologies

Adequate user authentication is a persistent problem, particularly with mobile devices, which tend to be highly personal and at the fringes of an organisation's influence. Yet these devices are being used increasingly in various business settings, where they pose a risk to security and privacy, not only from sensitive information they may contain, but also from the means they typically offer to access such information over wireless networks. User authentication is the first line of defence for a mobile device that falls into the hands of an unauthorised user. However, motivating users to enable simple password mechanisms and periodically update their authentication information is difficult at best. This paper examines some of the issues relating to the use of biometrics as a viable method of authentication on mobile wireless devices. It is also a critical analysis of some of the techniques currently employed and where appropriate, suggests novel hybrid ways in which they could be improved or modified. Both biometric technology and wireless setting based constraints that determine the feasibility and the performance of the authentication feature are specified. Some well known biometric technologies are briefly reviewed and their feasibility for wireless and mobile use is reviewed. Furthermore, a number of quantitative and qualitative parameters for evaluation are also presented. Biometric technologies are continuously advancing toward commercial implementation in wireless devices. When carefully designed and implemented, the advantage of biometric authentication arises mainly from increased convenience and coexistent improved security.

User Profile Management and Selection in Context-Aware Service Platforms for Networks Beyond 3G

In recent years, progress in the area of mobile telecommunications has changed our way of life, in the private as well as the business domain. Mobile and wireless networks have ever increasing bit rates, mobile network operators provide more and more services, and at the same time costs for the usage of mobile services and bit rates are decreasing. However, mobile services today still lack functions that seamlessly integrate into users’ everyday life. That is, service attributes such as context-awareness and personalisation are often either proprietary, limited or not available at all. In order to overcome this deficiency, telecommunications companies are heavily engaged in the research and development of service platforms for networks beyond 3G for the provisioning of innovative mobile services. These service platforms are to support such service attributes. Service platforms are to provide basic service-independent functions such as billing, identity management, context management, user profile management, etc. Instead of developing own solutions, developers of end-user services such as innovative messaging services or location-based services can utilise the platform-side functions for their own purposes. In doing so, the platform-side support for such functions takes away complexity, development time and development costs from service developers. Context-awareness and personalisation are two of the most important aspects of service platforms in telecommunications environments. The combination of context-awareness and personalisation features can also be described as situation-dependent personalisation of services. The support for this feature requires several processing steps. The focus of this doctoral thesis is on the processing step, in which the user’s current context is matched against situation-dependent user preferences to find the matching user preferences for the current user’s situation. However, to achieve this, a user profile management system and corresponding functionality is required. These parts are also covered by this thesis. Altogether, this thesis provides the following contributions: The first part of the contribution is mainly architecture-oriented. First and foremost, we provide a user profile management system that addresses the specific requirements of service platforms in telecommunications environments. In particular, the user profile management system has to deal with situation-specific user preferences and with user information for various services. In order to structure the user information, we also propose a user profile structure and the corresponding user profile ontology as part of an ontology infrastructure in a service platform. The second part of the contribution is the selection mechanism for finding matching situation-dependent user preferences for the personalisation of services. This functionality is provided as a sub-module of the user profile management system. Contrary to existing solutions, our selection mechanism is based on ontology reasoning. This mechanism is evaluated in terms of runtime performance and in terms of supported functionality compared to other approaches. The results of the evaluation show the benefits and the drawbacks of ontology modelling and ontology reasoning in practical applications.