Stakeholders and perspectivism in qualitative policy evaluation: A realist reflection

This paper examines the epistemological bases for the inclusion of stakeholders in policy research. While it concedes that the input of stakeholders provides essential expert and experiential knowledge for the understanding of complex policies and programmes, it contends that the approach which assumes that all interpretations of policy including those of stakeholders should be afforded equal validity, which we term relativist perspectivism, undermines the possibility of robust research by allowing power to replace methodological rigour as the primary research dynamic. It is noted that this problem tends to be more acute when the research is qualitative. A study into the gendered effects of Common Agricultural Policy reforms is used as an illustrative example of how research can be compromised by relativist perspectivism. It is argued that realist research methodologies uniquely provide the capacity to maintain epistemological robustness, while also being able to take due account of the perspectives of stakeholders.

Devolving Housing Policy and Practice in Northern Ireland 1998 - 2002

This paper reviews the effect of devolution on housing policy and practice in Northern Ireland. It outlines the history and context of devolution and housing policy in Northern Ireland, including the legacy and persistence of intense social conflict. Current devolution arrangements are reviewed, including the implications of enforced coalition for policy governance. The paper focuses on three dimensions of housing and housing-related policy development and implementation: social housing, especially the distinctive history and changing organisation of social housing provision; policies affecting the housing market, including the changing regime for spatial planning; and, regeneration and tenant participation. The paper argues that housing policy has tended to converge with policies in England, rather than moving towards a distinctively local agenda. Local political agendas remain dominated by disagreements over constitutional status, thus policy formulation is determined more by officials than by elected politicians.

Where there is smoke, there is (the potential for) fire: soft indicators of research and policy impact

There is a growing literature examining the impact of research on informing policy, and of research and policy on practice. Research and policy do not have the same types of impact on practice but can be evaluated using similar approaches. Sometimes the literature provides a platform for methodological debate but mostly it is concerned with how research can link to improvements in the process and outcomes of education, how it can promote innovative policies and practice, and how it may be successfully disseminated. Whether research-informed or research-based, policy and its implementation is often assessed on such 'hard' indicators of impact as changes in the number of students gaining five or more A to C grades in national examinations or a percentage fall in the number of exclusions in inner city schools. Such measures are necessarily crude, with large samples smoothing out errors and disguising instances of significant success or failure. Even when 'measurable' in such a fashion, however, the impact of any educational change or intervention may require a period of years to become observable. This paper considers circumstances in which short-term change may be implausible or difficult to observe. It explores how impact is currently theorized and researched and promotes the concept of 'soft' indicators of impact in circumstances in which the pursuit of conventional quantitative and qualitative evidence is rendered impractical within a reasonable cost and timeframe. Such indicators are characterized by their avowedly subjective, anecdotal and impressionistic provenance and have particular importance in the context of complex community education issues where the assessment of any impact often faces considerable problems of access. These indicators include the testimonies of those on whom the research intervention or policy focuses (for example, students, adult learners), the formative effects that are often reported (for example, by head teachers, community leaders) and media coverage. The collation and convergence of a wide variety of soft indicators (Where there is smoke …) is argued to offer a credible means of identifying subtle processes that are often neglected as evidence of potential and actual impact (… there is fire).

The policy context of leaving care services: A case study of Northern Ireland

The transition of foster youth from state care to independent living has received increased research, practice, and policy attention in the United States and in many other countries. Most contributions to this literature have focused on documenting poor outcomes across various dimensions of need in the young people's lives whereas little attention has been given to the policy context in which the responses to those needs are being developed. In this article, we argue that there is a pressing need for better understanding of how the policy context can both promote and impede the development of appropriate services. To illustrate our argument, we use Northern Ireland as a policy case study both because of recent initiatives underway there in regard to youth transitions from state care and because of the heightened political sensibilities associated with it as a society. We draw attention to the socio-political historical context, a number of intersecting social policies, and the place of social work as a key occupation involved in delivering service improvements. We conclude by suggesting that this case study not only highlights the need to address similar aspects of the policy on youth transition frorn state care in the United States but also demonstrates the benefits of reflecting on policy development and implementation elsewhere in the world.

Reconciling Children's Policy and Children's Rights: Barriers to Effective Government Delivery

Countries which have ratified the United Nations Convention on the Rights of the Child, have committed to implementing its principles in law and policy. This article explores the challenges for securing children's rights through policy, drawing on a research project conducted for the Northern Ireland Commissioner for Children and Young People, which sought to identify barriers to effective government delivery for children and young people from the perspective of key stakeholders. The research concluded that, while some barriers (such as delay and availability of data) are not child-specific, they can be accentuated when children and young people are the main focus of policy development and more so when seeking to adopt a child rights-compliant approach to policy development and implementation. 

Mandated prevention in child welfare: Considerations from a framework shaping ethical inquiry

Summary: This article outlines a framework for approaching ethical dilemmas arising from the development, evaluation and implementation of child welfare policies. As such, it is relevant to policy-makers, social researchers and social workers. The central tenets of the framework are developed by drawing on ideas from moral philosophy and critical social theory. These ideas are presented as axioms, theorems and corollaries, a format which has been employed in the social sciences to offer a rational justification for a set of claims. • Findings: This process of reasoning leads to four principle axioms that are seen to shape the ethical scrutiny of social policy: 1) problematizing knowledge; 2) utilizing structured forms of inquiry to enhance understanding; 3) engendering enabling communication with those affected by the ethical concern; and 4) enhancing self-awareness. • Applications: The four axioms are then applied, by way of example, to the current and contentious, 'third way' policy of mandated prevention in child welfare, where the aim is to obviate deleterious outcomes in later life. It is argued that the framework can be applied beyond this specific concern to other pressing, ethical challenges in child welfare.

Evaluating the social effects of the EU URBAN Community Initiative Programme

This article explores alternative interpretations of the meaning and method of urban policy evaluation within the European Union (EU) Structural Funds. Using the EU URBAN Community Initiative Programme 1994-1999 it draws a distinction between 'instrumental' techniques that are primarily concerned with performance and efficiency measures and 'interpretative' approaches that stress the need to explore power relationships in the development and delivery of spending programmes. Empirically, it reflects on the interpretation of EU guidance and the MEANS (Means for Evaluating Actions of a Structural Nature) Collection to evaluate the Derry/Londonderry (UK) URBAN Sub-programme 1994-1999. The analysis concludes by emphasizing the need to ensure that urban policy evaluation is consistent with the broader social turn in the scope and content of regeneration programmes.

Hiring Incentives and Labour Force Participation in Italy

A long-standing economic tradition maintains that labour supply reacts to market tightness; its sensitivity to job quality has received less attention. If firms hire workers with both temporary and open-ended contracts, does participation increase when more permanent jobs are available? We investigate this relationship within a policy evaluation framework; in particular, we examine how labour supply reacted in Italy to a recent subsidy in favour of open-ended contracts. This subsidy increased labour force participation by 1.4% in 2001 and 2.1% in 2002. This increase was concentrated on males aged 35-54, with a low or at most a secondary schooling level.

Efficacy and ethics:An investigation into the role of ethics, legitimacy and power in planning

The main thrust of the article is to consider the role of ethics, legitimacy, power and evidence-based policy in planning practice. The laboratory for the investigation is provided by developments in policymaking and implementation in the jurisdiction of Northern Ireland. In this context, each of the key themes is developed to establish a conceptual framework and the emerging issues are subsequently explored in an empirical investigation which deals with policy formulation and implementation, enabling lessons to be learnt about the motivations, tactics and strategies of the various participants in the process.

Northern Visions? Applying Q methodology to understand stakeholder views on the environmental and resource dimensions of sustainability

Q methodology was used to enable the identification of discourses among stakeholders to the environmental and resource dimensions of sustainability policies and to gain an understanding of the usefulness of Q methodology in informing sustainability policy development. The application of Q methodology has been useful in identifying shared discourses between different stakeholder groups, and providing insights into how stakeholders frame or understand policy issues; and recommendations are made for ongoing research priorities. These insights, in turn, informed the choice of scenarios for an in parallel process of policy evaluation using Ecological and Carbon Footprinting.

Which is the Only Game in Town? Minority Rights Issues in Estonia and Slovakia During and after EU Accession:Perspectives on European Politics and Society

Post-communist transition went hand in hand with the European integration process. Much of the literature on EU accession focuses on the rational decision to implement a set of European norms into domestic legislation pre-accession. It is often concluded that once EU membership is achieved, states succumb their rationality and act on the basis of internalised norms. The paper claims that the past literature overlooks the wider framework within which policy-makers operate before and after the accession, namely domestic sovereignty over policy-making and implementation. Tracing the policy dynamics in the area of minority rights in Estonia and Slovakia, we demonstrate that the European integration ushered greater domestic control over policy implementation on minority issues in two states exposed to a heavy dose of conditionality. As we observe, both states have consolidated their state- and nation-building policies referencing EU conditionality in the course of accession and later EU membership to assert centrality of domestic objectives for policy-making and implementation.

Strategic behaviour in choice experiments.

Practitioners of environmental economics sometimes use repeated trinary choice experiment surveys to estimate the value of environmental policies and programs for use in policy evaluation. These surveys have several advantages over simpler forms of non-market valuation: (1) researchers can estimate the marginal value of attributes of the good or service in question, making the results useful for benefits transfer; and (2) because respondents make several choices and choose from choice sets containing three options, efficiency of the willingness to pay estimate is improved over one-shot, binary choice formats. Despite these benefits, such surveys may have incentive properties which cause the resulting value estimates to be biased. This paper presents a theoretical demonstration that subjects often have an incentive to choose the second-best option in a repeated trinary choice survey. The model shows that due to the nature of factorial choice set design, the second-best option in the choice set will often be the status quo option. The paper reports a set of experiments designed to test these theoretical predictions in an induced-value setting. The experimental results are consistent with the theoretical predictions, demonstrating that repeated trinary choice experiment surveys can generate biased value estimates under a wide range of conditions.

Design and Development of a Film-based Intervention about Teenage Men and Unintended Pregnancy: Applying the Medical Research Council Framework in Practice

Following the UK Medical Research Council’s (MRC) guidelines for the development and evaluation of complex interventions, this study aimed to design, develop and optimise an educational intervention about young men and unintended teenage pregnancy based around an interactive film. The process involved identification of the relevant evidence base, development of a theoretical understanding of the phenomenon of unintended teenage pregnancy in relation to young men, and exploratory mixed methods research. The result was an evidence-based, theory-informed, user-endorsed intervention designed to meet the much neglected pregnancy education needs of teenage men and intended to increase both boys’ and girls’ intentions to avoid an unplanned pregnancy during adolescence. In prioritising the development phase, this paper addresses a gap in the literature on the processes of research-informed intervention design. It illustrates the application of the MRC guidelines in practice while offering a critique and additional guidance to programme developers on the MRC prescribed processes of developing interventions. Key lessons learned were: 1) know and engage the target population and engage gatekeepers in addressing contextual complexities; 2) know the targeted behaviours and model a process of change; and 3) look beyond development to evaluation and implementation.