Summary statistics, educational achievement gaps and the ecological fallacy

Summary statistics continue to play an important role in identifying and monitoring patterns and trends in educational inequalities between differing groups of pupils over time. However, this article argues that their uncritical use can also encourage the labelling of whole groups of pupils as ‘underachievers’ or ‘overachievers’ as the findings of group-level data are simply applied to individual group members, a practice commonly termed the ‘ecological fallacy’. Some of the adverse consequences of this will be outlined in relation to current debates concerning gender and ethnic differences in educational attainment. It will be argued that one way of countering this uncritical use of summary statistics and the ecological fallacy that it tends to encourage, is to make much more use of the principles and methods of what has been termed ‘exploratory data analysis’. Such an approach is illustrated through a secondary analysis of data from the Youth Cohort Study of England and Wales, focusing on gender and ethnic differences in educational attainment. It will be shown that, by placing an emphasis on the graphical display of data and on encouraging researchers to describe those data more qualitatively, such an approach represents an essential addition to the use of simple summary statistics and helps to avoid the limitations associated with them.

A critical review of some recent developments in quantitative research on gender and achievement in the United Kingdom

Over recent years the findings of a number of quantitative research studies have been published in the UK on gender and achievement. Much of this work has emanated from Stephen Gorard and his colleagues and has not only been highly critical of existing approaches to handling quantitative data but has also suggested a number of alternative and, what they claim to be, more valid ways of measuring differential patterns of achievement and underachievement between groups. This article shows how much of this work has been based upon rather under-developed measures of achievement and underachievement that tend, in turn, to generate a number of misleading findings that have questionable implications for practice. It will be argued that this body of work provides a useful case study in the problems of quantitative research that fails to engage adequately with the substantive theoretical and empirical literature and considers some of the implications of this for future research in this area.