Stakeholders and perspectivism in qualitative policy evaluation: A realist reflection

This paper examines the epistemological bases for the inclusion of stakeholders in policy research. While it concedes that the input of stakeholders provides essential expert and experiential knowledge for the understanding of complex policies and programmes, it contends that the approach which assumes that all interpretations of policy including those of stakeholders should be afforded equal validity, which we term relativist perspectivism, undermines the possibility of robust research by allowing power to replace methodological rigour as the primary research dynamic. It is noted that this problem tends to be more acute when the research is qualitative. A study into the gendered effects of Common Agricultural Policy reforms is used as an illustrative example of how research can be compromised by relativist perspectivism. It is argued that realist research methodologies uniquely provide the capacity to maintain epistemological robustness, while also being able to take due account of the perspectives of stakeholders.

Essays in partial identification and applications to treatment effects and policy evaluation.

Dans cette thèse, je me suis interessé à l’identification partielle des effets de traitements dans différents modèles de choix discrets avec traitements endogènes. Les modèles d’effets de traitement ont pour but de mesurer l’impact de certaines interventions sur certaines variables d’intérêt. Le type de traitement et la variable d’intérêt peuvent être défini de manière générale afin de pouvoir être appliqué à plusieurs différents contextes. Il y a plusieurs exemples de traitement en économie du travail, de la santé, de l’éducation, ou en organisation industrielle telle que les programmes de formation à l’emploi, les techniques médicales, l’investissement en recherche et développement, ou l’appartenance à un syndicat. La décision d’être traité ou pas n’est généralement pas aléatoire mais est basée sur des choix et des préférences individuelles. Dans un tel contexte, mesurer l’effet du traitement devient problématique car il faut tenir compte du biais de sélection. Plusieurs versions paramétriques de ces modèles ont été largement étudiées dans la littérature, cependant dans les modèles à variation discrète, la paramétrisation est une source importante d’identification. Dans un tel contexte, il est donc difficile de savoir si les résultats empiriques obtenus sont guidés par les données ou par la paramétrisation imposée au modèle. Etant donné, que les formes paramétriques proposées pour ces types de modèles n’ont généralement pas de fondement économique, je propose dans cette thèse de regarder la version nonparamétrique de ces modèles. Ceci permettra donc de proposer des politiques économiques plus robustes. La principale difficulté dans l’identification nonparamétrique de fonctions structurelles, est le fait que la structure suggérée ne permet pas d’identifier un unique processus générateur des données et ceci peut être du soit à la présence d’équilibres multiples ou soit à des contraintes sur les observables. Dans de telles situations, les méthodes d’identifications traditionnelles deviennent inapplicable d’où le récent développement de la littérature sur l’identification dans les modèles incomplets. Cette littérature porte une attention particuliere à l’identification de l’ensemble des fonctions structurelles d’intérêt qui sont compatibles avec la vraie distribution des données, cet ensemble est appelé : l’ensemble identifié. Par conséquent, dans le premier chapitre de la thèse, je caractérise l’ensemble identifié pour les effets de traitements dans le modèle triangulaire binaire. Dans le second chapitre, je considère le modèle de Roy discret. Je caractérise l’ensemble identifié pour les effets de traitements dans un modèle de choix de secteur lorsque la variable d’intérêt est discrète. Les hypothèses de sélection du secteur comprennent le choix de sélection simple, étendu et généralisé de Roy. Dans le dernier chapitre, je considère un modèle à variable dépendante binaire avec plusieurs dimensions d’hétérogéneité, tels que les jeux d’entrées ou de participation. je caractérise l’ensemble identifié pour les fonctions de profits des firmes dans un jeux avec deux firmes et à information complète. Dans tout les chapitres, l’ensemble identifié des fonctions d’intérêt sont écrites sous formes de bornes et assez simple pour être estimées à partir des méthodes d’inférence existantes.

A semiotic approach of multi-agent simulation for policy evaluation

Nowadays the changing environment becomes the main challenge for most of organizations, since they have to evaluate proper policies to adapt to the environment. In this paper, we propose a multi-agent simulation method to evaluate policies based on complex adaptive system theory. Furthermore, we propose a semiotic EDA (Epistemic, Deontic, Axiological) agent model to simulate agent's behavior in the system by incorporating the social norms reflecting the policy. A case study is also provided to validate our approach. Our research present better adaptability and validity than the qualitative analysis and experiment approach and the semiotic agent model provides high creditability to simulate agents' behavior.

Regional integration and public policy. Evaluation of the European experience and possible implications for Latin American integration

Includes bibliography

Rationalizing social policy: evaluation and viability

Includes bibliography

Essays in policy evaluation

This dissertation consists of three empirical studies that aim at providing new evidence in the field of public policy evaluation. In particular, the first two chapters focus on the effects of the European cohesion policy, while the third chapter assesses the effectiveness of Italian labour market incentives in reducing long-term unemployment. The first study analyses the effect of EU funds on life satisfaction across European regions , under the assumption that projects financed by structural funds in the fields of employment, education, health and environment may affect the overall quality of life in recipient regions. Using regional data from the European Social Survey in 2002-2006, it resorts to a regression discontinuity design, where the discontinuity is provided by the institutional framework of the policy. The second study aims at estimating the impact of large transfers from a centralized authority to a local administration on the incidence of white collar crimes. It merges a unique dataset on crimes committed in Italian municipalities between 2007 and 2011 with information on the disbursement of EU structural funds in 2007-2013 programming period, employing an instrumental variable estimation strategy that exploits the variation in the electoral cycle at local level. The third study analyses the impact of an Italian labour market policy that allowed firms to cut their labour costs on open-ended job contracts when hiring long-term unemployed workers. It takes advantage of a unique dataset that draws information from the unemployment lists in Veneto region and it resorts to a regression discontinuity approach to estimate the effect of the policy on the job finding rate of long-term unemployed workers.

Policy evaluation with travel behavior models : methodological issues and case studies.

Mode of access: Internet.

Applying a public policy evaluation methodology for local socio-economic impacts: an exploration in realistic modelling

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT

Assessing an experimental approach to industrial policy evaluation:Applying RCT+ to the case of Creative Credits

Experimental methods of policy evaluation are well-established in social policy and development eco-nomics but are rare in industrial and innovation policy. In this paper, we consider the arguments forapplying experimental methods to industrial policy measures, and propose an experimental policy eval-uation approach (which we call RCT+). This approach combines the randomised assignment of firmsto treatment and control groups with a longitudinal data collection strategy incorporating quantitativeand qualitative data (so-called mixed methods). The RCT+ approach is designed to provide a causativerather than purely summative evaluation, i.e. to assess both ‘whether’ and ‘how’ programme outcomesare achieved. In this paper, we assess the RCT+ approach through an evaluation of Creative Credits – aUK business-to-business innovation voucher initiative intended to promote new innovation partnershipsbetween SMEs and creative service providers. The results suggest the potential value of the RCT+ approachto industrial policy evaluation, and the benefits of mixed methods and longitudinal data collection.

Reframe QUT's evaluation framework : a case study of planning, policy and positioning leading to educational best practice

Every university in Australia has a set of policies that guide the institution in its educational practices, however, the policies are often developed in isolation to each other. Now imagine a space where policies are evidence-based, refined annually, cohesively interrelated, and meet stakeholders’ needs. Is this happenstance or the result of good planning? Culturally, Queensland University of Technology (QUT) is a risk-averse institution that takes pride in its financial solvency and is always keen to know “how are we going?” With a twenty-year history of annual reporting that assures the quality of course performance through multiple lines of evidence, QUT’s Learning and Teaching Unit went one step further and strategically aligned a suite of policies that take into consideration the needs of their stakeholders, collaborate with other areas across the institution and use multiple lines of evidence to inform curriculum decision-making. In QUT’s experience, strategic planning can lead to policy that is designed to meet stakeholders’ needs, not manage them; where decision-making is supported by evidence, not rhetoric; where all feedback is incorporated, not ignored; and where policies are cohesively interrelated, not isolated. While many may call this ‘policy nirvana’, QUT has positioned itself to demonstrate good educational practice through Reframe, its evaluation framework. In this case, best practice was achieved through the application of a theory of change and a design-led logic model that allows for transition to other institutions with different cultural specificity. The evaluation approach follows Seldin’s (2003) notion to offer depth and breadth to the evaluation framework along with Berk’s (2005) concept of multiple lines of evidence. In summary, this paper offers university executives, academics, planning and quality staff an opportunity to understand the critical steps that lead to strategic planning and design of evidence-based educational policy that positions a university for best practice in learning and teaching.