Where there is smoke, there is (the potential for) fire: soft indicators of research and policy impact

There is a growing literature examining the impact of research on informing policy, and of research and policy on practice. Research and policy do not have the same types of impact on practice but can be evaluated using similar approaches. Sometimes the literature provides a platform for methodological debate but mostly it is concerned with how research can link to improvements in the process and outcomes of education, how it can promote innovative policies and practice, and how it may be successfully disseminated. Whether research-informed or research-based, policy and its implementation is often assessed on such 'hard' indicators of impact as changes in the number of students gaining five or more A to C grades in national examinations or a percentage fall in the number of exclusions in inner city schools. Such measures are necessarily crude, with large samples smoothing out errors and disguising instances of significant success or failure. Even when 'measurable' in such a fashion, however, the impact of any educational change or intervention may require a period of years to become observable. This paper considers circumstances in which short-term change may be implausible or difficult to observe. It explores how impact is currently theorized and researched and promotes the concept of 'soft' indicators of impact in circumstances in which the pursuit of conventional quantitative and qualitative evidence is rendered impractical within a reasonable cost and timeframe. Such indicators are characterized by their avowedly subjective, anecdotal and impressionistic provenance and have particular importance in the context of complex community education issues where the assessment of any impact often faces considerable problems of access. These indicators include the testimonies of those on whom the research intervention or policy focuses (for example, students, adult learners), the formative effects that are often reported (for example, by head teachers, community leaders) and media coverage. The collation and convergence of a wide variety of soft indicators (Where there is smoke …) is argued to offer a credible means of identifying subtle processes that are often neglected as evidence of potential and actual impact (… there is fire).

Statin prescribing in Northern Ireland and England pre and post introduction of the quality and outcomes framework

Objective The objective of this research was to examine differences in patterns of statin prescribing between Northern Ireland and England both before and after the introduction of the Quality and Outcomes Framework (QOF). Setting: Primary care practices in Northern Ireland and England. Method Northern Ireland practices were matched with practices in England, statin prescribing data and QOF achievement scores (for the first year post-QOF) were obtained. Crude prescribing data from matched practices were manipulated to provide a data set of Defined Daily Doses (DDDs)/1,000 patients and cost/DDD/1,000 patients for each statin drug entity covering 1 year before and after the introduction of QOF. QOF achievements were converted into percentage scores for matched practices. Main outcome measure Cost per defined daily dose (DDD) per 1,000 patients. Results Significantly less statins (DDD/1,000 patients) were dispensed in Northern Ireland compared with the matched region in England both before and after the introduction of QOF (P

Prising open the black box: Critical realism, action research and social work

There is a growing interest in critical realism and its application to social work. This article makes a case for adopting this philosophical position in qualitative social work research. More specifically, it suggests that there is a concordance between critical realist premises and action research with its cyclical inquiry and advancement of social change. This combination of philosophy and method, it is argued, promotes anti-oppressive social work research and illuminates the processes shaping outcomes in programme evaluations. Overall, the article underscores the importance of 'depth' in qualitative inquiry by conceiving the social world in terms of five interlacing, social domains.

Adolescent men's attitudes in relation to pregnancy and pregnancy outcomes: A systematic review of the literature from 1980-2009

This review article reveals a long-standing gender bias in academic and policy research on adolescent pregnancy, which has led to the neglect of adolescent men's perspectives. The review summarizes the available literature on adolescent men's attitudes in relation to pregnancy occurrence and pregnancy outcomes in the context of addressing three questions: (1) What are adolescent men's attitudes to an adolescent pregnancy? (2) What are adolescent men's attitudes in relation to pregnancy outcomes? (3) What explanations are offered for the identified attitudes to adolescent pregnancy and resolution? The review establishes a foundation for future quantitative and qualitative research on adolescent men's perspectives. It emphasizes that a greater understanding of adolescent men's perspectives could lead to a re-framing of adolescent pregnancy away from being seen solely as a woman's issue. Furthermore, it is argued that the inclusion of adolescent men would lead to more effective adolescent pregnancy prevention and counseling programmes.

Action research and reflective practice: towards a holistic view

Two concepts that have captured the imagination of the educational community in the last 60 years have been those of ‘reflective practice’ and ‘action research’. Both, in their various forms, are considered to be critical dimensions of the professional development of teachers. However, whilst both were receiving academic attention during the 1930s and 1940s (Lewin, 1934, cited in Adelman, 1993; Lewin, 1946; Dewey, 1933), it was not until Stenhouse’s (1975) notion of the teacher-as-researcher that the two came most compellingly into relationship and educational action research as a process, which held at its centre different kinds of reflection, began to be reformulated in Britain (Carr, 1993). This article considers the important part played in teachers’ development by different kinds of action research. Its central thesis is that, although action research has a critical role to play not least as a means of building the capacity of teachers as researchers of their own practice, there has been insufficient attention given to both the nature of reflection in the action research process, and its relationship to the purposes, processes and outcomes. The article challenges the rational, cognitive models of reflection that are implicit in much of the action research literature. It suggests that more attention needs to be given to the importance of the role of emotion in understanding and developing the capacities for reflection which facilitates personal, professional and ultimately system change.

Learning and organization in the knowledge-based information economy: Initial findings from a participatory action research case study

This paper reports on an ongoing, multiphase, project-based action learning and research project. In particular, it summarizes some aspects of the learning climate and outcomes for a case study company In the software industry, Using a participatory action research approach, the learning company framework developed by Pedler et al, (1997) is used to initiate critical reflection in the company at three levels: managing director, senior management team and technical and professional staff. As such, this is one of the first systematic attempts to apply this framework to the entire organization and to a company in the knowledge-based learning economy. Two sets of issues are of general concern to the company: internal issues surrounding the company's reward and recognition policies and practices and the provision of accounting and control information in a business relevant way to all levels of staff; and external issues concerning the extent to which the company and its members actively learn from other companies and effectively capture, disseminate and use information accessed by staff in boundary-spanning roles. The paper concludes with some illustrations of changes being introduced by the company as a result of the feedback on and discussion of these issues.

Research priorities of adult intensive care nurses in 20 European countries: a Delphi study.

Aims: This paper is a report of a three round Delphi study of intensive care nursing research priorities in Europe (October 2006–April 2009).

Background: Internationally, priorities for research in intensive care nursing have received some attention focusing on healthcare interventions and patient needs. Studies as early as the 1980s identified priorities in the United States, United Kingdom, Hong Kong and Australia. Research priorities of intensive care nurses across the European Union are unknown.

Methods: The participants, invited in 2006, included 110 intensive care nurses, managers, educators and researchers from 20 European Critical Care Nursing Associations. Delphi round one was an emailed questionnaire inviting participants to list important areas for research. The list was content analysed and developed into an online questionnaire for rounds two and three. In round two, participants ranked the topics on a scale of 1–6 (not important to extremely important). Mean scores of round two were added to the questionnaire of round three and participants ranked the topics again.

Results: There were 52 research topics in 12 domains. There was a dominance of priorities in five main areas: patient safety; impact of evidence based practice on outcomes; impact of workforce on outcomes; well being of patients and relatives; and impact of end-of-life care on staff and practice.

Conclusions: The results reflect worldwide healthcare concerns and objectives and highlight topics that nurses view as fundamental to the care of critically ill patients. These topics provide a platform for future research efforts to improve clinical practice and care of patients in intensive care.

Developing outcomes for educational services: a children's rights-based approach

This article provides a rationale for and insight into an explicit children's rights-based approach to the identification of outcomes for proposed educational interventions. It presents a critical reflection on a research project which sought to integrate international children's rights standards into the design of services through a children's rights audit of potential outcomes and the meaningful engagement of children in the research and service design processes. While children are involved increasingly as co-researchers in qualitative studies, it is less common for this to occur in quantitative studies. This article offers some additional insight into children's participation in the interpretation of data from a large-scale baseline survey. The article concludes with an argument that international children's rights law provides not just a legal imperative but also a comprehensive framework with which to assert the case for increased recognition of children as salient stakeholders in all aspects of service design.

Treatment Foster Care for Improving Outcomes in Children and Young People: A Systematic Review

Objective: To assess the impact of treatment foster care (TFC) on psychosocial and behavioral outcomes, delinquency, placement stability, and discharge status for children and adolescents who, for reasons of severe medical, social, psychological and behavioural problems, were placed in out-of-home care in restrictive settings or at risk of placement in such settings. Method: Electronic bibliographic databases, web searches, and article reference lists were used to identify randomized controlled trials (RCTs) investigating the effectiveness of TFC with children and young people. The Cochrane Collaboration’s criteria were used to assess the methodological quality of studies that met the inclusion criteria. Wherever possible, extracted outcome data from similar studies were synthesized with random effects meta-analyses. Results: A total of 5 studies including 390 participants were included in this review. Data suggest that TFC may be a useful intervention for children and young people with complex emotional, psychological, and behavioural need, who are at risk of placements in nonfamily settings that restrict their liberty and opportunities for social inclusion. Conclusion: Although the inclusion criteria for this systematic review set a study design threshold higher than that of previous reviews, the findings mirror those of earlier reviews. While the results of individual studies generally indicate that TFC is a promising intervention for children and youth experiencing mental health problems, behavioral problems, or problems of delinquency, the evidence base is not robust and more research is needed due to the limited number of studies in this area.

General and Respiratory Health Outcomes in Adult Survivors of Bronchopulmonary Dysplasia: A Systematic Review

Background: The purpose of this systematic literature review was to examine current empirical research on general and respiratory health outcomes in adult survivors of bronchopulmonary dysplasia (BPD).

Methods: We searched seven databases up to the end of November 2010 (MEDLINE, PubMed, EMBASE, PsycINFO, Maternity and Infant Care, Cumulative Index of Nursing and Allied Health Literature, and Web of Knowledge). We independently screened and included only those studies concerning the assessment of outcome measures in adult survivors of BPD. Data on methodologic design and findings were extracted from each included study; in addition, the methodologic quality of each study was assessed using the Critical Appraisal Skills Programme checklist.

Results: Fourteen cohort studies met the review criteria. Of those, a total of eight studies were considered to be of high quality (score 9-12), five of moderate quality (score 5-8), and only one was of low quality (score 0-4). In all studies of adult survivors of BPD, differences were found between the index and control groups, suggesting that many adults survivors of BPD who were born preterm or with very low birth weight had more respiratory symptoms and pulmonary function abnormalities compared with their peers. Five studies concerning radiologic findings reported structural changes persisting into adulthood. Findings from three studies suggested impairment in exercise capacity, although firm conclusions were limited by the small sample size in the studies reviewed.

Conclusions: Compared with adults born at term, adult survivors of BPD have more impairment in general and respiratory health, which does not seem to diminish over time.