Managing patient deterioration: evaluation of what works.

Background: Rapid Response Systems (RRS) have been implemented nationally and internationally to improve patient safety in hospital. However, to date the majority of the RRS research evidence has focused on measuring the effectiveness of the intervention on patient outcomes. To evaluate RRS it has been recommended that a multimodal approach is required to address the broad range of process and outcome measures required to determine the effectiveness of the RRS concept. Aim: The aim of this paper is to evaluate the official RRS programme theoretical assumptions regarding how the programme is meant to work against actual practice in order to determine what works. Methods: The research design was a multiple case study approach of four wards in two hospitals in Northern Ireland. It followed the principles of realist evaluation research which allowed empirical data to be gathered to test and refine RRS programme theory [1]. This approach used a variety of mixed methods to test the programme theories including individual and focus group interviews with a purposive sample of 75 nurses and doctors, observation of ward practices and documentary analysis. The findings from the case studies were analysed and compared within and across cases to identify what works for whom and in what circumstances. Results: The RRS programme theories were critically evaluated and compared with study findings to develop a mid-range theory to explain what works, for whom in what circumstances. The findings of what works suggests that clinical experience, established working relationships, flexible implementation of protocols, ongoing experiential learning, empowerment and pre-emptive management are key to the success of RRS implementation.  Conclusion:These findings highlight the combination of factors that can improve the implementation of RRS and in light of this evidence several recommendations are made to provide policymakers with guidance and direction for their success and sustainability.References: 1.Pawson R and Tilley N. (1997) Realistic Evaluation. Sage Publications; LondonType of submission: Concurrent session Source of funding: Sandra Ryan Fellowship funded by the School of Nursing & Midwifery, Queen’s University of Belfast

Realistic evaluation of EWS and ALERT: factors enabling and constraining implementation.

Realistic Evaluation of EWS and ALERT: factors enabling and constraining implementation Background The implementation of EWS and ALERT in practice is essential to the success of Rapid Response Systems but is dependent upon nurses utilising EWS protocols and applying ALERT best practice guidelines. To date there is limited evidence on the effectiveness of EWS or ALERT as research has primarily focused on measuring patient outcomes (cardiac arrests, ICU admissions) following the implementation of a Rapid Response Team. Complex interventions in healthcare aimed at changing service delivery and related behaviour of health professionals require a different research approach to evaluate the evidence. To understand how and why EWS and ALERT work, or might not work, research needs to consider the social, cultural and organisational influences that will impact on successful implementation in practice. This requires a research approach that considers both the processes and outcomes of complex interventions, such as EWS and ALERT, implemented in practice. Realistic Evaluation is such an approach and was used to explain the factors that enable and constrain the implementation of EWS and ALERT in practice [1]. Aim The aim of this study was to evaluate factors that enabled and constrained the implementation and service delivery of early warnings systems (EWS) and ALERT in practice in order to provide direction for enabling their success and sustainability. Methods The research design was a multiple case study approach of four wards in two hospitals in Northern Ireland. It followed the principles of realist evaluation research which allowed empirical data to be gathered to test and refine RRS programme theory. This approach used a variety of mixed methods to test the programme theories including individual and focus group interviews, observation and documentary analysis in a two stage process. A purposive sample of 75 key informants participated in individual and focus group interviews. Observation and documentary analysis of EWS compliance data and ALERT training records provided further evidence to support or refute the interview findings. Data was analysed using NVIVO8 to categorise interview findings and SPSS for ALERT documentary data. These findings were further synthesised by undertaking a within and cross case comparison to explain the factors enabling and constraining EWS and ALERT. Results A cross case analysis highlighted similarities, differences and factors enabling or constraining successful implementation across the case study sites. Findings showed that personal (confidence; clinical judgement; personality), social (ward leadership; communication), organisational (workload and staffing issues; pressure from managers to complete EWS audit and targets), educational (constraints on training; no clinical educator on ward) and cultural (routine task delegated) influences impact on EWS and acute care training outcomes. There were also differences noted between medical and surgical wards across both case sites. Conclusions Realist Evaluation allows refinement and development of the RRS programme theory to explain the realities of practice. These refined RRS programme theories are capable of informing the planning of future service provision and provide direction for enabling their success and sustainability. References: 1. McGaughey J, Blackwood B, O’Halloran P, Trinder T. J. & Porter S. (2010) A realistic evaluation of Track and Trigger systems and acute care training for early recognition and management of deteriorating ward–based patients. Journal of Advanced Nursing 66 (4), 923-932. Type of submission: Concurrent session Source of funding: Sandra Ryan Fellowship funded by the School of Nursing & Midwifery, Queen’s University of Belfast

A realistic evaluation of Early Warning Systems and acute care training in Northern Ireland: findings and recommendations

Statement of purpose The purpose of this concurrent session is to present the main findings and recommendations from a five year study evaluating the implementation of Early Warning Systems (EWS) and the Acute Life-threatening Events: Recognition and Treatment (ALERT) course in Northern Ireland. The presentation will provide delegates with an understanding of those factors that enable and constrain successful implementation of EWS and ALERT in practice in order to provide an impetus for change. Methods The research design was a multiple case study approach of four wards in two hospitals in Northern Ireland. It followed the principles of realist evaluation research which allowed empirical data to be gathered to test and refine RRS programme theory [1]. The stages included identifying the programme theories underpinning EWS and ALERT, generating hypotheses, gathering empirical evidence and refining the programme theories. This approach used a variety of mixed methods including individual and focus group interviews, observation and documentary analysis of EWS compliance data and ALERT training records. A within and across case comparison facilitated the development of mid-range theories from the research evidence. Results The official RRS theories developed from the realist synthesis were critically evaluated and compared with the study findings to develop a mid-range theory to explain what works, for whom in what circumstances. The findings of what works suggests that clinical experience, established working relationships, flexible implementation of protocols, ongoing experiential learning, empowerment and pre-emptive management are key to the success of EWS and ALERT implementation. Each concept is presented as ‘context, mechanism and outcome configurations’ to provide an understanding of how the context impacts on individual reasoning or behaviour to produce certain outcomes. Conclusion These findings highlight the combination of factors that can improve the implementation and sustainability of EWS and ALERT and in light of this evidence several recommendations are made to provide policymakers with guidance and direction for future policy development. References: 1. Pawson R and Tilley N. (1997) Realistic Evaluation. Sage Publications; London Type of submission: Concurrent session Source of funding: Sandra Ryan Fellowship funded by the School of Nursing & Midwifery, Queen’s University of Belfast

Respecting and fulfilling the right of post-primary pupils to consent to participate in trials and evaluative research: A discussion paper

This paper provides an introduction to issues surrounding the participation rights of young people in research and the implications of their growing involvement in research as well as providing a discourse on the ethical implications related to consent. The unique contribution of this paper is that it considers children’s rights in respect to the increasing opportunities for young people to take part in evaluation research. The aim of this paper, therefore, is to acknowledge the growing involvement for young people in research and the implications of ensuring that their rights of participation are respected. Secondly we will consider the children’s rights legislation and our obligations as researchers to implement this. Finally we will explore consent as an issue in its own right as well as the practicalities of accessing participants. This paper will postulate that any research about young people should involve and prioritise at all stages of the research process; including participation in decision-making. We conclude by identifying five key principles, which we believe can help to facilitate the fulfilment of post-primary pupils’ ability to consent to participate in trials and evaluative research.

Predicting family caregiver psychosocial functioning in palliative care

Health professionals are expected to support family caregivers of patients requiring palliative care. However, there is a dearth of empirical evidence to help clinicians identify caregivers who might be at risk of poor psychosocial functioning.

Using mixed-methods to examine teachers’ attitudes to educational change:The case of the Skills for Life strategy for improving adult literacy and numeracy skills in England.

This paper reports on a longitudinal study that examines how a national reform introduced in England in the field of adult literacy, language, and numeracy is affecting teachers. The paper focuses on the use of a mixed methodology to explore teachers' attitudes to the reform and how these change over time. The quantitative strand includes the construction and use of a Likert-type instrument for measuring the attitudes of a panel of 1,500 teachers. The qualitative strand builds on the quantitative results and includes focus groups and in-depth interviews with a subsample of teachers in the panel. As the study is still in its initial phase, the purpose is not to present findings, but to discuss how quantitative and qualitative evidence can be combined in evaluation research.

Outcome measurement and service evaluation - a note on research design.

The purpose of this research note is to demonstrate how an individualised quality of life instrument could be adapted to provide a more accurate estimate of the impact of a social service on a person’s quality of life. An increase in quality of life between the start and end of a service is often taken as an indication that the service impacted positively on quality of life. The modifications to the quality of life instrument suggested in this paper show that this assumption is not always accurate and should be questioned directly.

On the Problems of Mixing RCTs with Qualitative Research: The Case of the MRC Framework for the Evaluation of Complex Healthcare Interventions

In the perceived hierarchy of research designs, the results from randomized controlled trials are considered to provide the highest level of evidence. Indeed these trials have been upheld as the gold standard in research. The benefits and limitations of the randomized controlled trial as a method of evaluating the effectiveness of healthcare interventions are presented. The article then examines the different levels of complexity within healthcare interventions and the problems this poses in determining effectiveness. In an effort to provide a solution to this problem, the Medical Research Council produced a framework to assist investigators to develop and evaluate complex healthcare interventions. The framework is described with reference to an example of implementing and evaluating protocols for weaning patients in the intensive care unit. The framework is critiqued on the basis that it involves an ambiguous or contradictory ontology, which fails to articulate the relationship between the positivism of randomized controlled trials with the relativism of qualitative approaches. It is concluded that the use of realist strategies in combination with randomized controlled trials provides the most coherent solution to this quandary

Microsatellite standardization and evaluation of genotyping error in a large multi-partner research programme for conservation of Atlantic salmon (Salmo salar L.).

Microsatellite genotyping is a common DNA characterization technique in population, ecological and evolutionary genetics research. Since different alleles are sized relative to internal size-standards, different laboratories must calibrate and standardize allelic designations when exchanging data. This interchange of microsatellite data can often prove problematic. Here, 16 microsatellite loci were calibrated and standardized for the Atlantic salmon, Salmo salar, across 12 laboratories. Although inconsistencies were observed, particularly due to differences between migration of DNA fragments and actual allelic size ('size shifts'), inter-laboratory calibration was successful. Standardization also allowed an assessment of the degree and partitioning of genotyping error. Notably, the global allelic error rate was reduced from 0.05 ± 0.01 prior to calibration to 0.01 ± 0.002 post-calibration. Most errors were found to occur during analysis (i.e. when size-calling alleles; the mean proportion of all errors that were analytical errors across loci was 0.58 after calibration). No evidence was found of an association between the degree of error and allelic size range of a locus, number of alleles, nor repeat type, nor was there evidence that genotyping errors were more prevalent when a laboratory analyzed samples outside of the usual geographic area they encounter. The microsatellite calibration between laboratories presented here will be especially important for genetic assignment of marine-caught Atlantic salmon, enabling analysis of marine mortality, a major factor in the observed declines of this highly valued species.