Where there is smoke, there is (the potential for) fire: soft indicators of research and policy impact

There is a growing literature examining the impact of research on informing policy, and of research and policy on practice. Research and policy do not have the same types of impact on practice but can be evaluated using similar approaches. Sometimes the literature provides a platform for methodological debate but mostly it is concerned with how research can link to improvements in the process and outcomes of education, how it can promote innovative policies and practice, and how it may be successfully disseminated. Whether research-informed or research-based, policy and its implementation is often assessed on such 'hard' indicators of impact as changes in the number of students gaining five or more A to C grades in national examinations or a percentage fall in the number of exclusions in inner city schools. Such measures are necessarily crude, with large samples smoothing out errors and disguising instances of significant success or failure. Even when 'measurable' in such a fashion, however, the impact of any educational change or intervention may require a period of years to become observable. This paper considers circumstances in which short-term change may be implausible or difficult to observe. It explores how impact is currently theorized and researched and promotes the concept of 'soft' indicators of impact in circumstances in which the pursuit of conventional quantitative and qualitative evidence is rendered impractical within a reasonable cost and timeframe. Such indicators are characterized by their avowedly subjective, anecdotal and impressionistic provenance and have particular importance in the context of complex community education issues where the assessment of any impact often faces considerable problems of access. These indicators include the testimonies of those on whom the research intervention or policy focuses (for example, students, adult learners), the formative effects that are often reported (for example, by head teachers, community leaders) and media coverage. The collation and convergence of a wide variety of soft indicators (Where there is smoke …) is argued to offer a credible means of identifying subtle processes that are often neglected as evidence of potential and actual impact (… there is fire).

Wellbeing, academic buoyancy and educational achievement in primary school students

The study explored the relationship between student wellbeing and academic achievement among 7–11 years old students and whether the relationship was moderated by gender and deprivation. 1081 students in Northern Ireland participated in a crosssectional survey that captured data on academic achievement and a range of wellbeing indicators. Findings suggested the existence of an underlying wellbeing factor, which was positively related to achievement. The relationship was not moderated by gender and/or deprivation. Findings were explored using a model of ‘academic buoyancy’. There was no evidence that suggested efforts to improve achievement that focus on wellbeing should be targeted speci?cally at students in economically deprived areas or be modi?ed in terms of gender.

Understanding Parental Influence on Educational Outcomes Among 9 Year Olds in Ireland: The Mediating Role of Resources, Attitudes and Children's Own Perspectives

The relationship between parental background and children's educational outcomes has been a dominant theme within the sociology of education. There has been an on-going debate as to the relative merits of explanations which focus on the role of socio-cultural reproduction and those which focus on rational choice. However, many empirical studies within the social stratification tradition fail to allow for children's own agency in shaping the relationship between social background and schooling outcomes. This paper draws on the first wave of a large-scale longitudinal study of over 8,000 nine-year-old children in Ireland, which combines information from parents, school principals, teachers and children themselves. Both social class and parental education are found to have significant effects on reading and mathematics test scores among nine year olds. These effects are partly mediated by home-based educational resources and activities, parents' educational expectations for their child, and parents' formal involvement in the school. More importantly, children's own engagement with, and attitudes to, school significantly influence their academic performance. The influence of children's own attitudes and actions can thus reinforce or mitigate the effect of social background factors. The analysis therefore provides a bridge between the large body of research on the intergenerational transmission of inequality and the emerging research and policy literature on children's rights.

Summary statistics, educational achievement gaps and the ecological fallacy

Summary statistics continue to play an important role in identifying and monitoring patterns and trends in educational inequalities between differing groups of pupils over time. However, this article argues that their uncritical use can also encourage the labelling of whole groups of pupils as ‘underachievers’ or ‘overachievers’ as the findings of group-level data are simply applied to individual group members, a practice commonly termed the ‘ecological fallacy’. Some of the adverse consequences of this will be outlined in relation to current debates concerning gender and ethnic differences in educational attainment. It will be argued that one way of countering this uncritical use of summary statistics and the ecological fallacy that it tends to encourage, is to make much more use of the principles and methods of what has been termed ‘exploratory data analysis’. Such an approach is illustrated through a secondary analysis of data from the Youth Cohort Study of England and Wales, focusing on gender and ethnic differences in educational attainment. It will be shown that, by placing an emphasis on the graphical display of data and on encouraging researchers to describe those data more qualitatively, such an approach represents an essential addition to the use of simple summary statistics and helps to avoid the limitations associated with them.

Constructions of locality and gender and their impact on the educational aspirations of working class children

Over recent years the moral panic that has surrounded 'boys' underachievement' has tended to encourage crude and essentialist comparisons between all boys and all girls and to eclipse the continuing and more profound effects on educational achievement exerted by social class and 'race'/ethnicity. While there are differences in educational achievement between working class boys and girls, these differences are relatively minor when comparing the overall achievement levels of working class children with those from higher, professional social class backgrounds. This paper argues that a need exists therefore for researchers to fully contextualise the gender differences that exist in educational achievement within the over-riding contexts provided by social class and 'race'/ethnicity. The paper provides an example of how this can be done through a case study of 11-year-old children from a Catholic, working class area in Belfast. The paper shows how the children's general educational aspirations are significantly mediated by their experiences of the local area in which they live. However, the way in which the children come to experience and construct a sense of locality differs between the boys and girls and this, it is argued, helps to explain the more positive educational aspirations held by some of the girls compared to the boys. The paper concludes by considering the relevance of locality for understanding its effects on educational aspirations among other working class and/or minority ethnic communities.

Teachers and pupils: educational experiences and school based responses to the conflict in Northern Ireland

This article describes a study which examined (a) the impact of the political conflict on teachers' and ppils' experiences of education in Northern Ireland and (b) the impact of curricular-based interventions designed to support the ppils and reduce prejudice. The focus of the second part of the article is on the prejudice reduction initiatives identified. A total of 44 staff and 78 pupils spread across 8 schools participated and both teachers' and ppils' perspectives were identified, the latter being an extremely important dimension which has rarely been addressed in previous studies of this area. The findings, which highlight the complexity of the impact of the political conflict, are considered to have both practical and theoretical implications for prejudice reduction programs.

Professional Educational Psychology and Community Relations Education in Northern Ireland.

This article provides an account of practice-based research of, at least, one-star quality in terms of its contribution to both theory and practice. Aimed at practitioner (as opposed to academic) psychologists, the article addresses a dimension of the practitioner role that has remained silent in the literature. The article makes creative and original connections between school effectiveness, school improvement and education in a divided society. Post 11th September, the article was described as being highly original, significant and relevant to all practising educational psychologists. Concrete evidence for this is gained from, eg: hits on the online electronic version (2002-2003 Annual Report of the Association of Educational Psychologists), citations in reviews of research, and author invitations to present his work at UK and international practitioner psychology conferences. The article is published in the premier journal reporting on quality applied educational research and practice within the United Kingdom and beyond.