The effectiveness of using patient-reported outcome measures as quality improvement tools

Aim: To investigate the value of using PROMs as quality improvement tools. Methods: Two systematic reviews were undertaken. The first reviewed the quantitative literature on the impact of PROMs feedback and the second reviewed the qualitative literature on the use of PROMs in practice. These reviews informed the focus of the primary research. A cluster randomised controlled trial (PROFILE) examined the impact of providing peer benchmarked PROMs feedback to consultant orthopaedic surgeons on improving outcomes for hip replacement surgery. Qualitative interviews with surgeons in the intervention arm of the trial examined the view of and reactions to the feedback. Results: The quantitative review of 17 studies found weak evidence to suggest that providing PROMs feedback to professionals improves patient outcomes. The qualitative review of 16 studies identified the barriers and facilitators to the use of PROMs based on four themes: practical considerations, attitudes towards the data, methodological concerns and the impact of feedback on care. The PROFILE trial included 11 surgeons and 215 patients in the intervention arm, and 10 surgeons and 217 patients in the control arm. The trial found no significant difference in the Oxford Hip Score between the arms (-0.7, 95% CI -1.9-0.5, p=0.2). Interviews with surgeons revealed mixed opinions about the value of the PROMs feedback and the information did not promote explicit changes to their practice. Conclusion: It is important to use PROMs which have been validated for the specific purpose of performance measurement, consult with professionals when developing a PROMs feedback intervention, communicate with professionals about the objectives of the data collection, educate professionals on the properties and interpretation of the data, and support professionals in using the information to improve care. It is also imperative that the burden of data collection and dissemination of the information is minimised.

Training in and assessment of the procedural skills required to perform peripheral nerve blockade

The training and ongoing education of medical practitioners has undergone major changes in an incremental fashion over the past 15 years. These changes have been driven by patient safety, educational, economic and legislative/regulatory factors. In the near future, training in procedural skills will undergo a paradigm shift to proficiency based progression with associated requirements for competence-based programmes, valid, reliable assessment tools and simulation technology. Before training begins, the learning outcomes require clear definition; any form of assessment applied should include measurement of these outcomes. Currently training in a procedural skill often takes place on an ad hoc basis. The number of attempts necessary to attain a defined degree of proficiency varies from procedure to procedure. Convincing evidence exists that simulation training helps trainees to acquire skills more efficiently rather than relying on opportunities in their clinical practice. Simulation provides a safe, stress free environment for trainees for skill acquisition, generalization and transfer via deliberate practice. The work described in this thesis contributes to a greater understanding of how medical procedures can be performed more safely and effectively through education. The effect of feedback, provided to novices in a standardized setting on a bench model, based on knowledge of performance was associated with an increase in the speed of skill acquisition and a decrease in error rate during initial learning. The timing of feedback was also associated with effective learning of skill. A marked attrition of skills (independent of the type of feedback provided) was demonstrable 24 hrs after they have first been learned. Using the principles of feedback as described above, when studying the effect of an intense training program on novices of varied years of experience in anaesthesia (i.e. the present training programmes / courses of an intense training day for one or more procedures). There was a marked attrition of skill at 24 hours with a significant correlation with increasing years of experience; there also appeared to be an inverse relationship between years of experience in anaesthesia and performance. The greater the number of years of practice experience, the longer it required a learner to acquire a new skill. The findings of the studies described in this thesis may have important implications for the trainers, trainees and training bodies in the design and implementation of training courses and the formats of delivery of changing curricula. Both curricula and training modalities will need to take account of characteristics of individual learners and the dynamic nature of procedural healthcare.

Nvidia, matrox, paper? - Undergraduate game design tools in the humanities and social sciences

Eschewing costly high-tech approaches, this paper looks at the experience of using low-tech approaches to game design assignments as problem based learning and assessment tool over a number of years in undergraduate teaching. General game design concepts are discussed, along with learning outcomes and assessment rubrics in line with Blooms Taxonomy based on evidence from students who had no prior experience of serious game play or design. Approaches to creating game design based assessments are offered.

Sex offender risk assessment and management in Ireland

This research examined sex offender risk assessment and management in Ireland. It focused on the statutory agencies with primary responsibility (Garda Síochána and the Probation Service). The goal was to document the historical, contextual and current systems, in addition to identifying areas of concern/improvements. The research was a mixed-methods approach. Eight studies were conducted. This incorporated documentary reviews of four Commission to Inquire Reports, qualitative interviews/focus groups with Garda staff, Probation Service staff, statutory agencies, community stakeholders, various Non-Governmental Organisations (NGOs) and sex offenders. Quantitative questionnaires were also administered to Garda staff. In all over 70 interviews were conducted and questionnaires were forwarded to 270 Garda members. The overall findings are: •Sex offender management in Ireland has become formal only since 2001. Knowledge, skills and expertise is in its infancy and is still evolving. •Mixed reviews and questions regarding fitness for purpose of currently used risk assessments tools were noted. •The Sex Offender Act 2001 requires additional elements to ensure safe sex offender monitoring and public protection. A judicial review of the Sex Offender Act 2001 was recommended by many respondents. •Interagency working under SORAM was hugely welcomed. The sharing of information has been welcomed by managing agencies as the key benefit to improving sex offender management. •Respondents reported that in practice, sex offender management in Ireland is fragmented and unevenly implemented. The research concluded that an independent National Sex Offender Authority should be established as an oversight and regulatory body for policy, strategy and direction in sex offender management. Further areas of research were also highlighted: ongoing evaluation and audits of the joint agency process and systems in place; recidivism studies tracking the risk assessment ratings and subsequent offending; and an evaluation of the current status of sex offender housing in Ireland.

Assessment in primary education in Ireland

This doctoral study examines assessment in primary education in the Republic of Ireland. The nature and purpose of assessment offer an insight into the values which are prioritised by an education system. In 2011, in the Republic of Ireland, the Department of Education and Skills (DES) published a strategy aiming to improve standards of literacy and numeracy. The document, entitled, Literacy and numeracy for learning and life: the national strategy to improve literacy and numeracy for children and young people 2011-2020, contains improvement targets as measured by standardised tests. It also mandates the increased use of standardised tests in primary education, and directs that aggregated scores should be reported to both Boards of Management and the DES. The study is framed by the theoretical perspectives of Michel Foucault and Pierre Bourdieu. Both of these commentators examine social policy and practice in an effort to provide insight into the history and operation of social institutions. This study is especially influenced by Foucault’s archaeology and genealogy of knowledge, and his notion of governmentality. It is also particularly cognisant of Bourdieu’s thoughts on habitus, doxa and capital. The study contains reviews of literature in the areas of assessment, assessment policy, and assessment policy in Ireland. These reviews highlight current debate in each of these areas while also grounding this debate in an historical context. The dissertation contains four empirical sections. 1) It analyses policy documents prepared in the development of the published strategy as well as investigating the strategy itself. In so doing it is aware of the burgeoning influence of pan-national bodies on policy development. 2) A number of high profile policy makers were interviewed as part of the study and their views are interpreted in light of the findings of the literature reviews. 3) The perspective of teachers was sought through a questionnaire survey. This gathered data on these teachers’ views on the purpose of assessment as well as their actual practice. 4) Finally, children were also included as participants in this study. They were interviewed in focus groups and encouraged to contribute drawings as well on their views of assessment in primary school. Literacy and numeracy for learning and life is seen as a seminal document in Irish education. This study is significant in its analysis of original data from high profile policy makers, including two Ministers for Education and Skills. It is also significant in its inclusion of the perspectives of primary school pupils. Finally, the study considers the nature and role of assessment in a holistic manner by including the views of policy makers, teachers and pupils. The study notes that policy development in Ireland underwent a change in the preparation of Literacy and numeracy for learning and life and that international influences, while present, are also mediated to suit the local context. It also highlights a lack of clarity in the definition of assessment in primary education and argues that there is a lack of balance in the approaches that are prioritised. The study demonstrates that teachers are impacted by the strategy but that they also change it by focusing on their own concerns while using assessment tools. The children provide compelling evidence of the impact of assessment on the learner. The study shows how assessment tools (and school subjects) are valued with differing levels of importance by a variety of stakeholders.