Realist evaluation: an immanent critique

This paper critically analyses realist evaluation, focussing on its primary analytical concepts: mechanisms, contexts, and outcomes. Noting that nursing investigators have had difficulty in operationalizing the concepts of mechanism and context, it is argued that their confusion is at least partially the result of ambiguities, inconsistencies, and contradictions in the realist evaluation model. Problematic issues include the adoption of empiricist and idealist positions, oscillation between determinism and voluntarism, subsumption of agency under structure, and categorical confusion between context and mechanism. In relation to outcomes, it is argued that realist evaluation's adoption of the fact/value distinction prevents it from taking into account the concerns of those affected by interventions. The aim of the paper is to use these immanent critiques of realist evaluation to construct an internally consistent realist approach to evaluation that is more amenable to being operationalized by nursing researchers.

‘Making it real’: evaluating the impact of service user and carer involvement in social work education

Service user and carer involvement (SUCI) in social work education in England is required by the profession’s regulator, the Health and Care Professions Council. However, a recent study of 83 HEIs in England reported that despite considerable progress in SUCI, there is no evidence that the learning derived from it is being transferred to social work practice. In this article we describe a study that examines the question: ‘What impact does SUCI have on the skills, knowledge and values of student social workers at the point of qualification and beyond?’ Students at universities in England and Northern Ireland completed online questionnaires and participated in focus groups, spanning a period immediately pre-qualification and between six to nine months post-qualification. From our findings, we identify four categories that influence the impact of service user involvement on students’ learning: student factors; service user and carer factors; programme factors; and practice factors; each comprises of a number of sub-categories. We propose that the model developed can be used by social work educators, service user and carer contributors and practitioners to maximise the impact of SUCI. We argue that our findings also have implications for employment-based learning routes and post-qualifying education.

Realistic evaluation of EWS and ALERT: factors enabling and constraining implementation.

Realistic Evaluation of EWS and ALERT: factors enabling and constraining implementation Background The implementation of EWS and ALERT in practice is essential to the success of Rapid Response Systems but is dependent upon nurses utilising EWS protocols and applying ALERT best practice guidelines. To date there is limited evidence on the effectiveness of EWS or ALERT as research has primarily focused on measuring patient outcomes (cardiac arrests, ICU admissions) following the implementation of a Rapid Response Team. Complex interventions in healthcare aimed at changing service delivery and related behaviour of health professionals require a different research approach to evaluate the evidence. To understand how and why EWS and ALERT work, or might not work, research needs to consider the social, cultural and organisational influences that will impact on successful implementation in practice. This requires a research approach that considers both the processes and outcomes of complex interventions, such as EWS and ALERT, implemented in practice. Realistic Evaluation is such an approach and was used to explain the factors that enable and constrain the implementation of EWS and ALERT in practice [1]. Aim The aim of this study was to evaluate factors that enabled and constrained the implementation and service delivery of early warnings systems (EWS) and ALERT in practice in order to provide direction for enabling their success and sustainability. Methods The research design was a multiple case study approach of four wards in two hospitals in Northern Ireland. It followed the principles of realist evaluation research which allowed empirical data to be gathered to test and refine RRS programme theory. This approach used a variety of mixed methods to test the programme theories including individual and focus group interviews, observation and documentary analysis in a two stage process. A purposive sample of 75 key informants participated in individual and focus group interviews. Observation and documentary analysis of EWS compliance data and ALERT training records provided further evidence to support or refute the interview findings. Data was analysed using NVIVO8 to categorise interview findings and SPSS for ALERT documentary data. These findings were further synthesised by undertaking a within and cross case comparison to explain the factors enabling and constraining EWS and ALERT. Results A cross case analysis highlighted similarities, differences and factors enabling or constraining successful implementation across the case study sites. Findings showed that personal (confidence; clinical judgement; personality), social (ward leadership; communication), organisational (workload and staffing issues; pressure from managers to complete EWS audit and targets), educational (constraints on training; no clinical educator on ward) and cultural (routine task delegated) influences impact on EWS and acute care training outcomes. There were also differences noted between medical and surgical wards across both case sites. Conclusions Realist Evaluation allows refinement and development of the RRS programme theory to explain the realities of practice. These refined RRS programme theories are capable of informing the planning of future service provision and provide direction for enabling their success and sustainability. References: 1. McGaughey J, Blackwood B, O’Halloran P, Trinder T. J. & Porter S. (2010) A realistic evaluation of Track and Trigger systems and acute care training for early recognition and management of deteriorating ward–based patients. Journal of Advanced Nursing 66 (4), 923-932. Type of submission: Concurrent session Source of funding: Sandra Ryan Fellowship funded by the School of Nursing & Midwifery, Queen’s University of Belfast

A realistic evaluation of track and trigger systems and acute care training for early recognition and treatment of deteriorating ward-based patients: research protocol.

Aim The aim of the study is to evaluate factors that enable or constrain the implementation and service delivery of early warnings systems or acute care training in practice. Background To date there is limited evidence to support the effectiveness of acute care initiatives (early warning systems, acute care training, outreach) in reducing the number of adverse events (cardiac arrest, death, unanticipated Intensive Care admission) through increased recognition and management of deteriorating ward based patients in hospital [1-3]. The reasons posited are that previous research primarily focused on measuring patient outcomes following the implementation of an intervention or programme without considering the social factors (the organisation, the people, external influences) which may have affected the process of implementation and hence measured end-points. Further research which considers the social processes is required in order to understand why a programme works, or does not work, in particular circumstances [4]. Method The design is a multiple case study approach of four general wards in two acute hospitals where Early Warning Systems (EWS) and Acute Life-threatening Events Recognition and Treatment (ALERT) course have been implemented. Various methods are being used to collect data about individual capacities, interpersonal relationships and institutional balance and infrastructures in order to understand the intended and unintended process outcomes of implementing EWS and ALERT in practice. This information will be gathered from individual and focus group interviews with key participants (ALERT facilitators, nursing and medical ALERT instructors, ward managers, doctors, ward nurses and health care assistants from each hospital); non-participant observation of ward organisation and structure; audit of patients' EWS charts and audit of the medical notes of patients who deteriorated during the study period to ascertain whether ALERT principles were followed. Discussion & progress to date This study commenced in January 2007. Ethical approval has been granted and data collection is ongoing with interviews being conducted with key stakeholders. The findings from this study will provide evidence for policy-makers to make informed decisions regarding the direction for strategic and service planning of acute care services to improve the level of care provided to acutely ill patients in hospital. References 1. Esmonde L, McDonnell A, Ball C, Waskett C, Morgan R, Rashidain A et al. Investigating the effectiveness of Critical Care Outreach Services: A systematic review. Intensive Care Medicine 2006; 32: 1713-1721 2. McGaughey J, Alderdice F, Fowler R, Kapila A, Mayhew A, Moutray M. Outreach and Early Warning Systems for the prevention of Intensive Care admission and death of critically ill patients on general hospital wards. Cochrane Database of Systematic Reviews 2007, Issue 3. www.thecochranelibrary.com 3. Winters BD, Pham JC, Hunt EA, Guallar E, Berenholtz S, Pronovost PJ (2007) Rapid Response Systems: A systematic review. Critical Care Medicine 2007; 35 (5): 1238-43 4. Pawson R and Tilley N. Realistic Evaluation. London; Sage: 1997

Protocol: A randomised controlled trial evaluation of Cancer Focus NI’s ‘Dead Cool’ smoking prevention programme in post-primary schools

This paper presents a research protocol for a randomised controlled efficacy trial of the ‘Dead Cool’ smoking prevention programme. Dead Cool is a three to four-hour programme designed to be used by teachers with Year 9 students in Northern Ireland. The main outcome of the programme is to prevent students from starting to smoke. The protocol reports a research design intended to test the efficacy of the programme in 20 post-primary school settings. Selected schools included those from secondary /grammar/integrated/single sex/coeducational, rural and urban schools from both the maintained and controlled state sector and independent sector schools. Outcome measures include self-reported behaviours, monitoring of carbon monoxide (CO) in exhaled breath and focus groups designed to assess implementation fidelity and opinions on efficacy in intervention schools and explore the ‘counterfactual’ potential treatments in control schools.

The association between myopia and various subtypes of lens opacity: SEE (Salisbury Eye Evaluation) project.

PURPOSE: To establish the relationship between myopia and lens opacity. DESIGN: Population-based cross-sectional study. PARTICIPANTS: Two thousand five hundred twenty participants from the Salisbury Eye Evaluation aged 65 to 84 years. METHODS: Participants filled out questionnaires regarding medical history, social habits, and a detailed history of distance spectacle wear. They underwent a full ocular examination. Lens photographs were taken for assessment of lens opacity using the Wilmer grading system. Multivariate logistic regression models using generalized estimating equations were used to analyze the relationship between lens opacity type and degree of myopia, while accounting for potential confounders. MAIN OUTCOME MEASURES: Presence of posterior subcapsular opacity, cortical opacity, or nuclear opacity. RESULTS: Significant associations were found between myopia and both nuclear and posterior subcapsular opacities. For nuclear opacity, the odds ratios (ORs) were 2.25 for myopia between -0.50 diopters (D) and -1.99 D (P<0.001), 3.65 for myopia between -2.00 D and -3.99 D (P<0.001), 4.54 for myopia between -4.00 D and -5.99 D (P<0.001), and 3.61 for myopia -6.00 D or more (P = 0.002). For posterior subcapsular cataracts, ORs were 1.59 for myopia between -0.50 D and -1.99 D (P = 0.11), 3.22 for myopia between -2.00 D and -3.99 D (P = 0.002), 5.36 for myopia between -4.00 D and -5.99 D (P<0.001), and 12.34 for myopia -6.00 D or more (P<0.001). No association was found between myopia and cortical opacity. The association between posterior subcapsular opacity and myopia was equally strong for those wearing glasses by age 21 years and for those without glasses; for nuclear opacity, significantly higher ORs were found for myopes who started wearing glasses after age 21. CONCLUSIONS: These results confirm the previously reported association between myopia, posterior subcapsular opacity, and nuclear opacity. Furthermore, the strong association between early spectacle wear and posterior subcapsular opacity among myopes, absent for nuclear opacity, suggests that myopia may precede opacity in the case of posterior subcapsular opacity, but not nuclear opacity. Measures of association between posterior subcapsular opacity and myopia were stronger in the current study than have previously been found. Longitudinal studies to confirm the association are warranted.

Adaptation and Evaluation of the Neighborhood Environment Walkability Scale in India (NEWS-India)

Physical inactivity is the fourth leading risk factor for global mortality, with most of these deaths occurring in low and middle-income countries (LMICs) like India. Research from developed countries has consistently demonstrated associations between built environment features and physical activity levels of populations. The development of culturally sensitive and reliable measures of the built environment is a necessary first step for accurate analysis of environmental correlates of physical activity in LMICs. This study systematically adapted the Neighborhood Environment Walkability Scale (NEWS) for India and evaluated aspects of test-retest reliability of the adapted version among Indian adults. Cultural adaptation of the NEWS was conducted by Indian and international experts. Semi-structured interviews were conducted with local residents and key informants in the city of Chennai, India. At baseline, participants (N = 370; female = 47.2%) from Chennai completed the adapted NEWS-India surveys on perceived residential density, land use mix-diversity, land use mix-access, street connectivity, infrastructure and safety for walking and cycling, aesthetics, traffic safety, and safety from crime. NEWS-India was administered for a second time to consenting participants (N = 62; female = 53.2%) with a gap of 2–3 weeks between successive administrations. Qualitative findings demonstrated that built environment barriers and constraints to active commuting and physical activity behaviors intersected with social ecological systems. The adapted NEWS subscales had moderate to high test-retest reliability (ICC range 0.48–0.99). The NEWS-India demonstrated acceptable measurement properties among Indian adults and may be a useful tool for evaluation of built environment attributes in India. Further adaptation and evaluation in rural and suburban settings in India is essential to create a version that could be used throughout India.

Internationalising the Social Work Curriculum: A Working Toolkit for Higher Education Facilities

This toolkit was collated by social work academics at Queen’s University Belfast (QUB), United Kingdom in collaboration with social work educators nationally and internationally who are committed to the internationalisation of the social work curriculum. The information and examples contained in this document offer a practical guide for the development and evaluation of international opportunities for students and academics. The resources were designed with diverse educational settings in mind for the enhancement of social work learning and teaching.

A prospective comparative evaluation of a Criminal Justice Liaison and Diversion Service in Belfast

There is continued interest in the planning, development and implementation of services designed to identify, detainees with mental illness and connect them to health and social services. However, currently little is known about how best to configure, organise and deliver these services. The study employed a prospective follow-up design with a comparator group to describe and evaluate a police mental health liaison service based in Belfast. Participants were recruited from two neighbouring police stations, only one of which provided a mental health liaison service. Outcomes including mental health status, drug and alcohol misuse, risk-related behaviour and ‘administrative’ outcomes were assessed at the time of arrest and six months later. The service was successful in identifying and assessing detainees though there appeared to be similar between-group levels of mental health problems over time. Results highlight a need to develop firmer linkages and pathways between criminal justice liaison / diversion services and routine health and social services. 

An Evaluation of the Drug and Alcohol Service User Network in NI

Objectives:
The process evaluation will consider the views of the appointed SUN workers and representatives from selected service user groups as regards the setting up and maintenance of the SUN network. This component of the evaluation will also examine the perceptions of stakeholders from a number of relevant organisations.

The outcome evaluation will assess the effectiveness of the SUN project in achieving the intended outcomes as outlined in the original Action Plans.
The following outcomes will be evaluated:
To ascertain the level to which the SUN has provided support, information and advice to existing service user groups.
To examine the SUN co-ordination of Trust and regional networks of service user groups.
To consider how the SUN assists organisations to establish and maintain service user groups.
To examine the level of current and future membership of service users on relevant groups, with a particular focus on engagement of hard to reach populations.
To gauge service user perceptions of the Service User Network.
To examine the levels of training provided and consider the efficacy of training.

Process Evaluation of the Roots of Empathy Program

This paper presents the findings of a qualitative process evaluation of the Roots of Empathy (ROE) programme. ROE is a universal, classroom-based intervention, which aims to enhance social and emotional learning of primary (elementary) school children. Effective delivery of such complex social interventions in real-world settings requires in-depth knowledge and understanding of factors that interact to influence implementation and fidelity. A case study methodology was employed with six schools, to explore the views of key actors and stakeholders involved in the delivery and receipt of the programme. Overall, ROE was delivered with high fidelity and the programme was viewed positively across the schools. However, one issue was the varied level of interest and awareness of the programme from parents.

Ansiedade social e assertividade na adolescência

O presente trabalho pretende caracterizar a associação entre ansiedade social e assertividade na adolescência, bem como desenvolver um modelo compreensivo e de intervenção teórica e empiricamente fundamentado acerca desta problemática. A ansiedade social é definida por medo intenso em situações sociais, associado a crenças negativas predisponentes e comportamentos de segurança ou evitamento subsequentes de situações sociais. A assertividade consiste numa resposta comportamental de auto-expressão empática, fundamentada em crenças positivas e activação emocional reduzida. Assim, estes dois conceitos parecem estar em dissonância, uma vez que, por definição, a ansiedade social surge associada a défice de comportamentos sociais adequados e a assertividade a diminuída activação ansiosa em eventos sociais. Por outro lado, ansiedade social e défice assertivo poderão fundamentar-se em mecanismos psicológicos semelhantes. Para verificar estas duas premissas, o presente trabalho utilizou uma amostra de 679 adolescentes do ensino secundário público de ambos os sexos. Para avaliar a ansiedade social nas três dimensões do funcionamento psicológico foram utilizadas a Escala de Crenças e Pensamentos Sociais e a Escala de Ansiedade e Evitamento de Situações Sociais para Adolescentes. No caso da assertividade, foram utilizados o Questionário de Esquema Interpessoal Assertivo e Escala de Comportamento Interpessoal. A análise de dados permite verificar a existência de uma associação recíproca negativa entre ansiedade social e assertividade, em todos os níveis considerados. Igualmente, os resultados obtidos indicam que esta associação poderá ser fundamentada na existência de baixas crenças sociais positivas que activam pensamentos sociais negativos e subsequentemente ansiedade e desconforto em situações sociais e reduzida frequência de comportamento assertivo. Esta conclusão fundamentou uma intervenção integrada para a promoção da gestão de ansiedade social e da prática de competências assertivas. Esta intervenção foi construída, implementada e avaliada em dois ensaios clínicos junto a 6 adolescentes. Os resultados de significância clínica indicam que o programa tem eficácia terapêutica, ainda que este estudo preliminar não exclua a necessidade de uma avaliação mais aprofundada do benefício associado a esta intervenção. Estes trabalhos assumem, assim, implicações educativas e terapêuticas, ao permitir explicitar e clarificar a associação entre ansiedade social e assertividade, e ao contribuir para o desenvolvimento e avaliação de formas de intervenção adequadas junto ao adolescente social tímido ou inibido. A compreensão e intervenção preventiva para a promoção do ajustamento psicossocial do adolescente emergem como uma realidade possível, pertinente e acessível a todos os agentes educativos.

Social-aware opportunistic routing

The increased capabilities (e.g., processing, storage) of portable devices along with the constant need of users to retrieve and send information have introduced a new form of communication. Users can seamlessly exchange data by means of opportunistic contacts among them and this is what characterizes the opportunistic networks (OppNets). OppNets allow users to communicate even when an end-to-end path may not exist between them. Since 2007, there has been a trend to improve the exchange of data by considering social similarity metrics. Social relationships, shared interests, and popularity are examples of such metrics that have been employed successfully: as users interact based on relationships and interests, this information can be used to decide on the best next forwarders of information. This Thesis work combines the features of today's devices found in the regular urban environment with the current social-awareness trend in the context of opportunistic routing. To achieve this goal, this work was divided into di erent tasks that map to a set of speci c objectives, leading to the following contributions: i) an up-to-date opportunistic routing taxonomy; ii) a universal evaluation framework that aids in devising and testing new routing proposals; iii) three social-aware utility functions that consider the dynamic user behavior and can be easily incorporated to other routing proposals; iv) two opportunistic routing proposals based on the users' daily routines and on the content traversing the network and interest of users in such content; and v) a structure analysis of the social-based network formed based on the approaches devised in this work.

Contributos para a eficácia do clustering usando o tagging social

Nos últimos anos temos vindo a assistir a uma mudança na forma como a informação é disponibilizada online. O surgimento da web para todos possibilitou a fácil edição, disponibilização e partilha da informação gerando um considerável aumento da mesma. Rapidamente surgiram sistemas que permitem a coleção e partilha dessa informação, que para além de possibilitarem a coleção dos recursos também permitem que os utilizadores a descrevam utilizando tags ou comentários. A organização automática dessa informação é um dos maiores desafios no contexto da web atual. Apesar de existirem vários algoritmos de clustering, o compromisso entre a eficácia (formação de grupos que fazem sentido) e a eficiência (execução em tempo aceitável) é difícil de encontrar. Neste sentido, esta investigação tem por problemática aferir se um sistema de agrupamento automático de documentos, melhora a sua eficácia quando se integra um sistema de classificação social. Analisámos e discutimos dois métodos baseados no algoritmo k-means para o clustering de documentos e que possibilitam a integração do tagging social nesse processo. O primeiro permite a integração das tags diretamente no Vector Space Model e o segundo propõe a integração das tags para a seleção das sementes iniciais. O primeiro método permite que as tags sejam pesadas em função da sua ocorrência no documento através do parâmetro Social Slider. Este método foi criado tendo por base um modelo de predição que sugere que, quando se utiliza a similaridade dos cossenos, documentos que partilham tags ficam mais próximos enquanto que, no caso de não partilharem, ficam mais distantes. O segundo método deu origem a um algoritmo que denominamos k-C. Este para além de permitir a seleção inicial das sementes através de uma rede de tags também altera a forma como os novos centróides em cada iteração são calculados. A alteração ao cálculo dos centróides teve em consideração uma reflexão sobre a utilização da distância euclidiana e similaridade dos cossenos no algoritmo de clustering k-means. No contexto da avaliação dos algoritmos foram propostos dois algoritmos, o algoritmo da “Ground truth automática” e o algoritmo MCI. O primeiro permite a deteção da estrutura dos dados, caso seja desconhecida, e o segundo é uma medida de avaliação interna baseada na similaridade dos cossenos entre o documento mais próximo de cada documento. A análise de resultados preliminares sugere que a utilização do primeiro método de integração das tags no VSM tem mais impacto no algoritmo k-means do que no algoritmo k-C. Além disso, os resultados obtidos evidenciam que não existe correlação entre a escolha do parâmetro SS e a qualidade dos clusters. Neste sentido, os restantes testes foram conduzidos utilizando apenas o algoritmo k-C (sem integração de tags no VSM), sendo que os resultados obtidos indicam que a utilização deste algoritmo tende a gerar clusters mais eficazes.