Identifying criteria for the assessment of pharmacy students' communication skills with patients

Objectives To identify criteria by which patients can assess the communication skills of pharmacy students. Method Potential assessment criteria were generated from 2 main sources: a literature review and a focus group discussion. A modified two-round Delphi survey was subsequently conducted with 35 professionals who were actively involved in teaching and assessing communication skills of pharmacy students to determine the importance and reliability of each criterion. Results Consensus ratings identified 7 criteria that were important measures of pharmacy students' communication skills and could be reliably assessed by patients. Conclusions A modified two-round Delphi consultation survey successfully identified criteria that can be used by patients to assess the communication skills of pharmacy undergraduates. Future work will examine the feasibility of using patients as assessors of communication skills of pharmacy students, preregistration pharmacists, and qualified pharmacists.

An evaluation of Home Office extended interviews for police personnel

In this thesis the validity of an Assessment Centre (called 'Extended Interview') operated on behalf of the British police is investigated. This Assessment Centre (AC) is used to select from amongst internal candidates (serving policemen and policewomen) and external candidates (graduates) for places on an accelerated promotion scheme. The literature is reviewed with respect to history, content, structure, reliability, validity, efficiency and usefulness of ACs, and to contextual issues surrounding AC use. The history of, background to and content of police Extended Interviews (Els) is described, and research issues are identified. Internal validation involved regression of overall EI grades on measures from component tests, exercises, interviews and peer nominations. Four samples numbering 126, 73, 86 and 109 were used in this part of the research. External validation involved regression of three types of criteria - training grades, rank attained, and supervisory ratings - on all EI measures. Follow-up periods for job criteria ranged from 7 to 19 years. Three samples, numbering 223, 157 and 86, were used in this part of the research. In subsidiary investigations, supervisory ratings were factor analysed and criteria intercorrelated. For two of the samples involved in the external validition, clinical/judgemental prediction was compared with mechanical (unit-weighted composite) prediction. Main conclusions are that: (1) EI selection decisions were valid, but only for a job performance criterion; relatively low validity overall was interpreted principally in terms of the questionable job relatedness of the EI procedure; (2) Els as a whole had more validity than was reflected in final EI decisions; (3) assessors' use of information was not optimum, tending to over-emphasize subjectively derived information particularly from interviews; and (4) mechanical prediction was superior to clinical/judgemental prediction for five major criteria.

The mosaic test:benchmarking colour-based image retrieval systems using image mosaics

Evaluation and benchmarking in content-based image retrieval has always been a somewhat neglected research area, making it difficult to judge the efficacy of many presented approaches. In this paper we investigate the issue of benchmarking for colour-based image retrieval systems, which enable users to retrieve images from a database based on lowlevel colour content alone. We argue that current image retrieval evaluation methods are not suited to benchmarking colour-based image retrieval systems, due in main to not allowing users to reflect upon the suitability of retrieved images within the context of a creative project and their reliance on highly subjective ground-truths. As a solution to these issues, the research presented here introduces the Mosaic Test for evaluating colour-based image retrieval systems, in which test-users are asked to create an image mosaic of a predetermined target image, using the colour-based image retrieval system that is being evaluated. We report on our findings from a user study which suggests that the Mosaic Test overcomes the major drawbacks associated with existing image retrieval evaluation methods, by enabling users to reflect upon image selections and automatically measuring image relevance in a way that correlates with the perception of many human assessors. We therefore propose that the Mosaic Test be adopted as a standardised benchmark for evaluating and comparing colour-based image retrieval systems.

Using XML and XSLT for flexible elicitation of mental-health risk knowledge

Current tools for assessing risks associated with mental-health problems require assessors to make high-level judgements based on clinical experience. This paper describes how new technologies can enhance qualitative research methods to identify lower-level cues underlying these judgements, which can be collected by people without a specialist mental-health background. Content analysis of interviews with 46 multidisciplinary mental-health experts exposed the cues and their interrelationships, which were represented by a mind map using software that stores maps as XML. All 46 mind maps were integrated into a single XML knowledge structure and analysed by a Lisp program to generate quantitative information about the numbers of experts associated with each part of it. The knowledge was refined by the experts, using software developed in Flash to record their collective views within the XML itself. These views specified how the XML should be transformed by XSLT, a technology for rendering XML, which resulted in a validated hierarchical knowledge structure associating patient cues with risks. Changing knowledge elicitation requirements were accommodated by flexible transformations of XML data using XSLT, which also facilitated generation of multiple data-gathering tools suiting different assessment circumstances and levels of mental-health knowledge. © 2007 Informa UK Ltd All rights reserved.

Understanding data collection behaviour of mental health practitioners

Failure to detect patients at risk of attempting suicide can result in tragic consequences. Identifying risks earlier and more accurately helps prevent serious incidents occurring and is the objective of the GRiST clinical decision support system (CDSS). One of the problems it faces is high variability in the type and quantity of data submitted for patients, who are assessed in multiple contexts along the care pathway. Although GRiST identifies up to 138 patient cues to collect, only about half of them are relevant for any one patient and their roles may not be for risk evaluation but more for risk management. This paper explores the data collection behaviour of clinicians using GRiST to see whether it can elucidate which variables are important for risk evaluations and when. The GRiST CDSS is based on a cognitive model of human expertise manifested by a sophisticated hierarchical knowledge structure or tree. This structure is used by the GRiST interface to provide top-down controlled access to the patient data. Our research explores relationships between the answers given to these higher-level 'branch' questions to see whether they can help direct assessors to the most important data, depending on the patient profile and assessment context. The outcome is a model for dynamic data collection driven by the knowledge hierarchy. It has potential for improving other clinical decision support systems operating in domains with high dimensional data that are only partially collected and in a variety of combinations.

An analysis and comparison of commonly available United Kingdom prescribing resources

Background and objective: Safe prescribing requires accurate and practical information about drugs. Our objective was to measure the utility of current sources of prescribing guidance when used to inform practical prescribing decisions, and to compare current sources of prescribing guidance in the UK with idealized prescribing guidance. Methods: We developed 25 clinical scenarios. Two independent assessors rated and ranked the performance of five common sources of prescribing guidance in the UK when used to answer the clinical scenarios. A third adjudicator facilitated review of any disparities. An idealized list of contents for prescribing guidance was developed and sent for comments to academics and users of prescribing guidance. Following consultation an operational check was used to assess compliance with the idealized criteria. The main outcome measures were relative utility in answering the clinical scenarios and compliance with the idealized prescribing guidance. Results: Current sources of prescribing guidance used in the UK differ in their utility, when measured using clinical scenarios. The British National Formulary (BNF) and EMIS LV were the best performing sources in terms of both ranking [mean rank 1·24 and 2·20] and rating [%excellent or adequate 100% and 72%]. Current sources differed in the extent to which they fulfilled criteria for ideal prescribing guidance, but the BNF, and EMIS LV to a lesser extent, closely matched the criteria. Discussion: We have demonstrated how clinical scenarios can be used to assess prescribing guidance resources. Producers of prescribing guidance documents should consider our idealized template. Prescribers require high-quality information to support their practice. Conclusion: Our test was helpful in distinguishing between prescribing resources. Producers of prescribing guidance should consider the utility of their products to end-users, particularly in those more complex areas where prescribers may need most support. Existing UK prescribing guidance resources differ in their ability to provide assistance to prescribers. © 2010 Blackwell Publishing Ltd.

Introducing a pilot data collection model for real-time evaluation of data redundancy

In order to reduce serious health incidents, individuals with high risks need to be identified as early as possible so that effective intervention and preventive care can be provided. This requires regular and efficient assessments of risk within communities that are the first point of contacts for individuals. Clinical Decision Support Systems CDSSs have been developed to help with the task of risk assessment, however such systems and their underpinning classification models are tailored towards those with clinical expertise. Communities where regular risk assessments are required lack such expertise. This paper presents the continuation of GRiST research team efforts to disseminate clinical expertise to communities. Based on our earlier published findings, this paper introduces the framework and skeleton for a data collection and risk classification model that evaluates data redundancy in real-time, detects the risk-informative data and guides the risk assessors towards collecting those data. By doing so, it enables non-experts within the communities to conduct reliable Mental Health risk triage.