The 'thin descriptions' of the secondary analyses of PISA

The heavy hammer methods of OECD and PISA in influencing policy through the rankings and through its policy advice are well documented. This speculative paper explores the more subtle and perhaps deeper implications of the development of the PISA database, and of the secondary analysis that is performed using this database. Speculating with concepts from Science and Technology Studies, this paper suggests that PISA deflates “ontologically luxuriant objects” into “ontologically impoverished objects” through standardization and simplification. Freed from their moorings and translated into inscriptions, these ontologically impoverished objects are promiscuous, freely combining with other such objects across spaces and times in different ways to produce lessons for policy and practice. In this paper, I suggest that, while these promiscuous relations may produce mathematically defensible assertions, such findings may be ontologically absurd. Using data from interviews with measurement and policy experts, as well as published secondary analyses, this paper ventures some speculative ideas about how we might understand the PISA database and the use of this database in secondary analysis. The paper argues that secondary analysis is not merely a mathematical or technical exercise but a sociotechnical one, and that, given its influence and reach, it attempts to open up the black boxes of the PISA database and the practices of secondary analysis, and make them available for wider sociological and philosophical examination and critique.