Applying a public policy evaluation methodology for local socio-economic impacts: an exploration in realistic modelling

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT

Assessing an experimental approach to industrial policy evaluation:Applying RCT+ to the case of Creative Credits

Experimental methods of policy evaluation are well-established in social policy and development eco-nomics but are rare in industrial and innovation policy. In this paper, we consider the arguments forapplying experimental methods to industrial policy measures, and propose an experimental policy eval-uation approach (which we call RCT+). This approach combines the randomised assignment of firmsto treatment and control groups with a longitudinal data collection strategy incorporating quantitativeand qualitative data (so-called mixed methods). The RCT+ approach is designed to provide a causativerather than purely summative evaluation, i.e. to assess both ‘whether’ and ‘how’ programme outcomesare achieved. In this paper, we assess the RCT+ approach through an evaluation of Creative Credits – aUK business-to-business innovation voucher initiative intended to promote new innovation partnershipsbetween SMEs and creative service providers. The results suggest the potential value of the RCT+ approachto industrial policy evaluation, and the benefits of mixed methods and longitudinal data collection.

GPs' perspectives on managing the NHS Health Check in primary care:a qualitative evaluation of implementation in one area of England

OBJECTIVES: To evaluate the implementation of the National Health Service (NHS) Health Check programme in one area of England from the perspective of general practitioners (GPs). DESIGN: A qualitative exploratory study was conducted with GPs and other healthcare professionals involved in delivering the NHS Health Check and with patients. This paper reports the experience of GPs and focuses on the management of the Heath Check programme in primary care. SETTING: Primary care surgeries in the Heart of Birmingham region (now under the auspices of the Birmingham Cross City Clinical Commissioning Group) were invited to take part in the larger scale evaluation. This study focuses on a subset of those surgeries whose GPs were willing to participate. PARTICIPANTS: 9 GPs from different practices volunteered. GPs served an ethnically diverse region with areas of socioeconomic deprivation. Ethnicities of participant GPs included South Asian, South Asian British, white, black British and Chinese. METHODS: Individual semistructured interviews were conducted with GPs face to face or via telephone. Thematic analysis was used to analyse verbatim transcripts. RESULTS: Themes were generated which represent GPs' experiences of managing the NHS Health Check: primary care as a commercial enterprise; 'buy in' to concordance in preventive healthcare; following protocol and support provision. These themes represent the key issues raised by GPs. They reveal variability in the implementation of NHS Health Checks. GPs also need support in allocating resources to the Health Check including training on how to conduct checks in a concordant (or collaborative) way. CONCLUSIONS: The variability observed in this small-scale evaluation corroborates existing findings suggesting a need for more standardisation. Further large-scale research is needed to determine how that could be achieved. Work needs to be done to further develop a concordant approach to lifestyle advice which involves tailored individual goal setting rather than a paternalistic advice-giving model.

The curious absence of inter-municipal co-operation in England

In Europe local authorities often work with their neighbouring municipalities, whether to address a specific task or goal or through the course of regular policy making and implementation. In England, however, inter-municipal co-operation (IMC) is less common. Councils may work with service providers from the private and non-profit sectors but less often with neighbouring local authorities. Why this is the case may be explained by a number of historical and policy factors that often encourage councils to compete, rather than to work collaboratively with each other. The present government has encouraged councils to work in partnership with other organizations but there are few examples of increased horizontal cooperation between local authorities. Instead the prevailing model remains fixed on vertical co-working predicated on a principal-agent relationship between higher and lower tiers of government.

International handbook on privatization

Privatization has dominated industrial restructuring programs since the 1980s and continues to do so. This authoritative and accessible Handbook considers all aspects of this key issue, including the theory of privatization; privatization in transition, developed and developing economics; as well the economic regulation of privatized industries. The studies in this volume, introduced by international experts in the field 'presents evidence of the scope and effects of privatization, and consequently provide the basis for improving both policy formulation and implementation. However, they also emphasize that privatization is not an end in itself. It is argued that for privatization to be worthwhile and for lasting economic efficiency gains to be achieved, supporting reforms must accompany most privatization programs, particularly in the arenas of corporate governance and capital markets, product market competition, and state regulatory processes. Furthermore, several contributions demonstrate that the degree to which ownership and market liberalization can be usefully separated, and whether privatization without either competition or effective regulation is worthwhile, remain controversial issues. Furnishing the reader with a comprehensive and lively discussion of privatization in theory and practice, this Handbook will be the essential source of information for researchers in the field, and for a wide-ranging audience including public policy makers and specialists, development experts and agencies, international banks, public policy and regulation economists, and management consultants.

Policy issues in rural areas:an examination with special reference to Cumbria

This thesis develops the requirements of the Cumbria Division of the MAFF to have detailed information on a number of rural topics of particular concern to the area's socia-economic advisory service. Information was generated upon the effects of road developments upon agriculture; the possibility of economic and employment growth through tourism, industry, forestry and agriculture; and upon their relationship with conservation and development control issues generally. A working conference was organised (The Whitbarrow Exercise) to review in specific terms a number of the above problems, in which representatives of the major groups active in rural policy formulation and implementation participated. The study was extended to consider these policy issues on a more prosperous agricultural estate; and in the county of Cumbria as a whole. An examination of the development and likely future impact of agricultural policy upon rural policy generally was also undertaken. All the research was set in the context of an extensive literature review. The results indicate that while state intervention to relieve those problems collectively known as rural deprivation still has an important place in modern rural policy, the scope for such intervention to be successful is limited. Opportunities for employment and wealth creation through tourism, forestry, industry and agriculture are limited for social and economic reasons; developments in these sectors can have adverse effects upon the environment; can compound existing problems; and are often resisted by local people. The lack of success of such ventures indicates continued structural change within rural communities, with some adverse effects for the less privileged members. Recognising this it is argued that fural policy seeks to adapt to, rather than attempt to fundamentally alter inevitable change, recognising that in the long term social and structural problems will resolve themselves. It is further argued that a reduction in state support for agriculture appears inevitable, and this can bring considerable conservation benefits, even in upland areas where positive links between agriculture and conservation have been found by some commentators. It is also argued that for social and economic reasons, and because of the declining importance of agricultural land, a vigorous landscape and ecological conservation policy is pursued by planning authorities and is reasonable. With regard to road developments on agricultural land, the research has shawn that although it is the norm far the agricultural community to experience severe difficulty during developments, these can be overcame by increasing the resources of professional expertise available to affected farmers. This indicates a possible important increased role for the MAFF in the development process.

ICT adoption and ebusiness development:Understanding ICT adoption amongst ethnic minority businesses

This paper builds upon a series of studies that have identified the comparatively low uptake of ICT amongst EMBs (Ram and Smallbone, 1999; Foley and Ram 2002; Allinson et al., 2004). Existing studies have only tentatively considered the causal factors for this lower adoption rate in comparison to non-EMBs. Within the context of a pilot study, aiming to understand ICT adoption amongst EMBs, an action research approach is adopted as a means of influencing, evaluating and underpinning ICT adoption (Beckinsale and Ram, 2006). The approach is methodologically distinctive in its utilization of action research with a critical realist approach as a means of policy evaluation; this is conducive to 'policy learning', and the understanding of causal mechanisms in the EMBs. Actors involved in the implementation were interviewed, as well as the business owners themselves. Another distinguishing feature is the application of Yap et al.'s (1992) schema to identify causal mechanisms, to examine actions and experiences and to understand the impact on the EMBs studied. The findings highlight the complex interaction and relationships of internal and external factors in shaping approaches to ICT implementation. Factors such as size, strategy and business age are germane but so too, albeit to a lesser degree, are cultural influences such as involvement in co-ethnic networks. Significantly, changes to particular causal mechanisms such as business support altered adoption and implementation of ICT. Distinct issues for Chinese and Asian-owned business and potentially more broadly EMBs are identified providing a framework for other EMBs. © The Author(s) 2010.

The Long-Term impact of Business Support? - Exploring the Role of Evaluation Timing using Micro Data

The original contribution of this work is threefold. Firstly, this thesis develops a critical perspective on current evaluation practice of business support, with focus on the timing of evaluation. The general time frame applied for business support policy evaluation is limited to one to two, seldom three years post intervention. This is despite calls for long-term impact studies by various authors, concerned about time lags before effects are fully realised. This desire for long-term evaluation opposes the requirements by policy-makers and funders, seeking quick results. Also, current ‘best practice’ frameworks do not refer to timing or its implications, and data availability affects the ability to undertake long-term evaluation. Secondly, this thesis provides methodological value for follow-up and similar studies by using data linking of scheme-beneficiary data with official performance datasets. Thus data availability problems are avoided through the use of secondary data. Thirdly, this thesis builds the evidence, through the application of a longitudinal impact study of small business support in England, covering seven years of post intervention data. This illustrates the variability of results for different evaluation periods, and the value in using multiple years of data for a robust understanding of support impact. For survival, impact of assistance is found to be immediate, but limited. Concerning growth, significant impact centres on a two to three year period post intervention for the linear selection and quantile regression models – positive for employment and turnover, negative for productivity. Attribution of impact may present a problem for subsequent periods. The results clearly support the argument for the use of longitudinal data and analysis, and a greater appreciation by evaluators of the factor time. This analysis recommends a time frame of four to five years post intervention for soft business support evaluation.

The long-term impact of business support? - Exploring the role of evaluation timing using micro data

The original contribution of this work is threefold. Firstly, this thesis develops a critical perspective on current evaluation practice of business support, with focus on the timing of evaluation. The general time frame applied for business support policy evaluation is limited to one to two, seldom three years post intervention. This is despite calls for long-term impact studies by various authors, concerned about time lags before effects are fully realised. This desire for long-term evaluation opposes the requirements by policy-makers and funders, seeking quick results. Also, current ‘best practice’ frameworks do not refer to timing or its implications, and data availability affects the ability to undertake long-term evaluation. Secondly, this thesis provides methodological value for follow-up and similar studies by using data linking of scheme-beneficiary data with official performance datasets. Thus data availability problems are avoided through the use of secondary data. Thirdly, this thesis builds the evidence, through the application of a longitudinal impact study of small business support in England, covering seven years of post intervention data. This illustrates the variability of results for different evaluation periods, and the value in using multiple years of data for a robust understanding of support impact. For survival, impact of assistance is found to be immediate, but limited. Concerning growth, significant impact centres on a two to three year period post intervention for the linear selection and quantile regression models – positive for employment and turnover, negative for productivity. Attribution of impact may present a problem for subsequent periods. The results clearly support the argument for the use of longitudinal data and analysis, and a greater appreciation by evaluators of the factor time. This analysis recommends a time frame of four to five years post intervention for soft business support evaluation.

Accessibility, deprivation and rural planning policies.

Since the Second World War a range of policies have been implemented by central and local government agencies, with a view to improving accessibility to facilities, housing and employment opportunities within rural areas. It has been suggested that a lack of reasonable access to a range of such facilities and opportunities constitutes a key aspect of deprivation or disadvantage for rural residents. Despite considerable interest, very few attempts have been made to assess the nature and incidence of this disadvantage or the reaction of different sections of the population of rural areas to it. Moreover, almost all previous assessments have relied on so-called 'objective' measures of accessibility and disadvantage and failed to consider the relationship between such measures and 'subjective' measures such as individual perceptions. It is this gap in knowledge that the research described in this thesis has addressed. Following a critical review of relevant literature the thesis describes the way in which data on 'objective' and 'subjective' indicators of accessibility and behavioural responses to accessibility problems was collected, in six case study areas in Shropshire. Analysis of this data indicates that planning and other government policies have failed to significantly improve rural resident's accessibility to their basic requirements, and may in some cases have exacerbated it, and that as a result certain sections of the rural population are relatively disadvantaged. Moreover, analysis shows that .certain aspects of individual subjective' assessments of such accessibility disadvantage are significantly associated with more easily-obtained 'objective' measures. By using discriminant analysis the research demonstrates that it is possible to predict the likely levels of satisfaction with access to facilities from a range of 'objective' measures. The research concludes by highlighting the potential practical applications of such indicators in policy formulation, policy appraisal and policy evaluation.

The value of proprietary health and safety software to proactive health and safety management

There has been little research in health and safety management concernmg the application of information technology to the field. This thesis attempts to stimulate interest in this area by analysing the value of proprietary health and safety software to proactive health and safety management. The thesis is based upon the detailed software evaluation of seven pieces of proprietary health and safety software. It features a discussion concerning the development of information technology and health and safety management, a review of the key issues identified during the software evaluations, an analysis of the commercial market for this type of software, and a consideration of the broader issues which surround the use of this software. It also includes practical guidance for the evaluation, selection, implementation and maintenance of all health and safety management software. This includes a comprehensive software evaluation chart. The implications of the research are considered for proprietary health and safety software, the application of information technology to health and safety management, and for future research.

Policy into practice:a study of public policy implementation and the role of learning

Public policy becomes managerial practice through a process of implementation. There is an established literature within Implementation Studies which explains the variables and some of the processes involved in implementation, but less attention has been focused upon how public service managers convert new policy initiatives into practice. The research proposes that managers and their organisations have to go through a process of learning in order to achieve the implementation of public policy. Data was collected over a five year period from four case studies of capital investment appraisal in the British National Health Service. Further data was collected from taped interviews by key actors within the case studies. The findings suggest that managers do learn to implement policy and four factors are important in this learning process. These are; (i) the nature of bureaucratic responsibility; (ii) the motivation of actors towards learning; (iii) the passage of time which allows for the development of competence and (iv) the use of project team structures. The research has demonstrated that the conversion of policy into practice occurs through the operationalisation of solutions to policy problems via job tasks. As such it suggests that in understanding how policy is implemented, technical learning is more important than cultural learning, in this context. In conclusion, a "Model of Learned Implementation" is presented, together with a discussion of some of the implications of the research. These are the possible use of more pilot projects for new policy initiatives and the more systematic diffusion of knowledge about implementation solutions.