Where there is smoke, there is (the potential for) fire: soft indicators of research and policy impact

There is a growing literature examining the impact of research on informing policy, and of research and policy on practice. Research and policy do not have the same types of impact on practice but can be evaluated using similar approaches. Sometimes the literature provides a platform for methodological debate but mostly it is concerned with how research can link to improvements in the process and outcomes of education, how it can promote innovative policies and practice, and how it may be successfully disseminated. Whether research-informed or research-based, policy and its implementation is often assessed on such 'hard' indicators of impact as changes in the number of students gaining five or more A to C grades in national examinations or a percentage fall in the number of exclusions in inner city schools. Such measures are necessarily crude, with large samples smoothing out errors and disguising instances of significant success or failure. Even when 'measurable' in such a fashion, however, the impact of any educational change or intervention may require a period of years to become observable. This paper considers circumstances in which short-term change may be implausible or difficult to observe. It explores how impact is currently theorized and researched and promotes the concept of 'soft' indicators of impact in circumstances in which the pursuit of conventional quantitative and qualitative evidence is rendered impractical within a reasonable cost and timeframe. Such indicators are characterized by their avowedly subjective, anecdotal and impressionistic provenance and have particular importance in the context of complex community education issues where the assessment of any impact often faces considerable problems of access. These indicators include the testimonies of those on whom the research intervention or policy focuses (for example, students, adult learners), the formative effects that are often reported (for example, by head teachers, community leaders) and media coverage. The collation and convergence of a wide variety of soft indicators (Where there is smoke …) is argued to offer a credible means of identifying subtle processes that are often neglected as evidence of potential and actual impact (… there is fire).

Prising open the black box: Critical realism, action research and social work

There is a growing interest in critical realism and its application to social work. This article makes a case for adopting this philosophical position in qualitative social work research. More specifically, it suggests that there is a concordance between critical realist premises and action research with its cyclical inquiry and advancement of social change. This combination of philosophy and method, it is argued, promotes anti-oppressive social work research and illuminates the processes shaping outcomes in programme evaluations. Overall, the article underscores the importance of 'depth' in qualitative inquiry by conceiving the social world in terms of five interlacing, social domains.

Action research and reflective practice: towards a holistic view

Two concepts that have captured the imagination of the educational community in the last 60 years have been those of ‘reflective practice’ and ‘action research’. Both, in their various forms, are considered to be critical dimensions of the professional development of teachers. However, whilst both were receiving academic attention during the 1930s and 1940s (Lewin, 1934, cited in Adelman, 1993; Lewin, 1946; Dewey, 1933), it was not until Stenhouse’s (1975) notion of the teacher-as-researcher that the two came most compellingly into relationship and educational action research as a process, which held at its centre different kinds of reflection, began to be reformulated in Britain (Carr, 1993). This article considers the important part played in teachers’ development by different kinds of action research. Its central thesis is that, although action research has a critical role to play not least as a means of building the capacity of teachers as researchers of their own practice, there has been insufficient attention given to both the nature of reflection in the action research process, and its relationship to the purposes, processes and outcomes. The article challenges the rational, cognitive models of reflection that are implicit in much of the action research literature. It suggests that more attention needs to be given to the importance of the role of emotion in understanding and developing the capacities for reflection which facilitates personal, professional and ultimately system change.