Assessing the effectiveness of business support services in England:evidence from a theory based evaluation

In England, publicly supported advice to small firms is organized primarily through the Business Link (BL) network. Using the programme theory underlying this business support, we develop four propositions and test these empirically using data from a new survey of over 3000 English SMEs. We find strong support for the value to BL operators of a high profile to boost take-up. We find support for the BL’s market segmentation that targets intensive assistance to younger firms and those with limited liability. Allowing for sample selection, we find no significant effects on growth from ‘other’ assistance but find a significant employment boost from intensive assistance. This partially supports the programme theory assertion that BL improves business growth and strongly supports the proposition that there are differential outcomes from intensive and other assistance. This suggests an improvement in the BL network, compared with earlier studies, notably Roper et al. (2001), Roper and Hart (2005).

An evaluation of certain aspects of an occupational health service

This thesis has been concerned with obtaining evidence to explore the proposition that the provision of occupational health services as arranged at the present time represents a misallocation of resources. The research has been undertaken within the occupational health service of a large Midlands food factory. As the research progressed it became evident that questions were being raised about the nature and scope of occupational health as well as the contribution, in combating danger at work, that occupational health services can make to the health and safety team. These questions have been scrutinized in depth, as they are clearly important, and a resolution of the problem of the definition of occupational health has been proposed. I have taken the approach of attempting to identify specific objectives or benefits of occupational health activities so that it is possible to assess how far these objectives are being achieved. I have looked at three aspects of occupational health; audiometry, physiotherapy and pre-employment medical examinations as these activities embody crucial concepts which are common to all activities in an occupational health programme. A three category classification of occupational health activities is proposed such that the three activities provide examples within each category. These are called personnel therapy, personnel input screening and personnel throughput screening. I conclude that I have not shown audiometry to be cost-effective. My observations of the physiotherapy service lead me to support the suggestion that there is a decline in sickness absence rates due to physiotherapy in industry. With pre-employment medical examinations I have shown that the service is product safety oriented and that benefits are extremely difficult to identify. In regard to the three services studied, in the one factory investigated, and because of the immeasurability of certain activities, I find support for the proposition that the mix of occupational health services as provided at the present time represents a misallocation of resources.

Evaluation and development of drainage and pipeline construction processes

In 1974 Dr D M Bramwell published his research work at the University of Aston a part of which was the establishment of an elemental work study data base covering drainage construction. The Transport and Road Research Laboratory decided to, extend that work as part of their continuing research programme into the design and construction of buried pipelines by placing a research contract with Bryant Construction. This research may be considered under two broad categories. In the first, site studies were undertaken to validate and extend the data base. The studies showed good agreement with the existing data with the exception of the excavation trench shoring and pipelaying data which was amended to incorporate new construction plant and methods. An inter-active on-line computer system for drainage estimating was developed. This system stores the elemental data, synthesizes the standard time of each drainage operation and is used to determine the required resources and construction method of the total drainage activity. The remainder of the research was into the general topic of construction efficiency. An on-line command driven computer system was produced. This system uses a stochastic simulation technique, based on distributions of site efficiency measurements to evaluate the effects of varying performance levels. The analysis of this performance data quantities the variability inherent in construction and demonstrates how some of this variability can be reconciled by considering the characteristics of a contract. A long term trend of decreasing efficiency with contract duration was also identified. The results obtained from the simulation suite were compared to site records collected from current contracts. This showed that this approach will give comparable answers, but these are greatly affected by the site performance parameters.

Reflection and reflexivity in reviewing and evaluating CDIO:an empirical approach to evaluation

Facilitated by an Engineer and a Social Scientist, both of whom have expertise in Engineering Education Research and Evaluation (EERE), this interactive workshop is divided into three main sections, each one focusing on a different area of evaluation. It will build on research conducted at Aston University School of Engineering and Applied Science to explore and critique the value of introducing CDIO across the first year undergraduate curriculum. Participants will be invited to consider the pedagogical and engineering related challenges of evaluating the academic and practical value of CDIO as a strategy for learning and teaching in the discipline. An empirical approach to evaluation developed by the researchers to provide empirically grounded evidence of the pedagogical and vocational value of CDIO will form the theoretical and conceptual basis of the workshop. This approach is distinctive in that it encapsulates both engineering and social science methods of evaluation. It is also contemporaneous in nature, with the researchers acting as a ‘fly on the wall’ capturing data as the programme unfolds. Through facilitated discussion and participation, the workshop will provide colleagues with the opportunity to develop a cross-disciplinary, empirically grounded research proposal specifically for the purposes of critically evaluating CDIO. It is anticipated that during the workshop, colleagues will work together in small groups. Suitable pedagogical approaches and tools will be suggested and a purposefully developed Engineering Education Research Guide, written by the workshop facilitators, will be given to all participants to inform and support the Workshop approach.

Modelling in the evaluation of a manufacturing strategy

This thesis describes research that has developed the principles of a modelling tool for the analytical evaluation of a manufacturing strategy. The appropriate process of manufacturing strategy formulation is based on mental synthesis with formal planning processes supporting this role. Inherent to such processes is a stage where the effects of alternative strategies on the performance of a manufacturing system must be evaluated so that a choice of preferred strategy can be made. Invariably this evaluation is carried out by practitioners applying mechanisms of judgement, bargaining and analysis. Ibis thesis makes a significant and original contribution to the provision of analytical support for practitioners in this role. The research programme commences by defining the requirements of analytical strategy evaluation from the perspective of practitioners. A broad taxonomy of models has been used to identify a set of potentially suitable techniques for the strategy evaluation task. Then, where possible, unsuitable modelling techniques have been identified on the basis of evidence in the literature and discarded from this set. The remaining modelling techniques have been critically appraised by testing representative contemporary modelling tools in an industrially based experimentation programme. The results show that individual modelling techniques exhibit various limitations in the strategy evaluation role, though some combinations do appear to provide the necessary functionality. On the basis of this comprehensive and in-depth knowledge a modelling tool ' has been specifically designed for this task. Further experimental testing has then been conducted to verify the principles of this modelling tool. Ibis research has bridged the fields of manufacturing strategy formulation and manufacturing systems modelling and makes two contributions to knowledge. Firstly, a comprehensive and in-depth platform of knowledge has been established about modelling techniques in manufacturing strategy evaluation. Secondly, the principles of a tool that supports this role have been formed and verified.

Description and preliminary results from a structured specialist behavioural weight management group intervention:Specialist Lifestyle Management (SLiM) programme

Background -  Specialist Lifestyle Management (SLiM) is a structured patient education and self-management group weight management programme. Each session is run monthly over a 6-month period providing a less intensive long-term approach. The groups are patient-centred incorporating educational, motivational, behavioural and cognitive elements. The theoretical background, programme structure and preliminary results of SLiM are presented. Subjects/methods - The study was a pragmatic service evaluation of obese patients with a body mass index (BMI) ≥35 kg/m2 with comorbidity or ≥40 kg/m2 without comorbidity referred to a specialist weight management service in the West Midlands, UK. 828 patients were enrolled within SLiM over a 48-month period. Trained facilitators delivered the programme. Preliminary anonymised data were analysed using the intention-to-treat principle. The primary outcome measure was weight loss at 3 and 6 months with comparisons between completers and non-completers performed. The last observation carried forward was used for missing data. Results - Of the 828 enrolled within SLiM, 464 completed the programme (56%). The mean baseline weight was 135 kg (BMI=49.1 kg/m2) with 87.2% of patients having a BMI≥40 kg/m2 and 12.4% with BMI≥60 kg/m2. The mean weight change of all patients enrolled was −4.1 kg (95% CI −3.6 to −4.6 kg, p=0.0001) at the end of SLiM, with completers (n=464) achieving −5.5 kg (95% CI −4.2 to −6.2 kg, p=0.0001) and non-completers achieving −2.3 kg (p=0.0001). The majority (78.6%) who attended the 6-month programme achieved weight loss with 32.3% achieving a ≥5% weight loss. Conclusions - The SLiM programme is an effective group intervention for the management of severe and complex obesity.

Assessing an experimental approach to industrial policy evaluation:Applying RCT+ to the case of Creative Credits

Experimental methods of policy evaluation are well-established in social policy and development eco-nomics but are rare in industrial and innovation policy. In this paper, we consider the arguments forapplying experimental methods to industrial policy measures, and propose an experimental policy eval-uation approach (which we call RCT+). This approach combines the randomised assignment of firmsto treatment and control groups with a longitudinal data collection strategy incorporating quantitativeand qualitative data (so-called mixed methods). The RCT+ approach is designed to provide a causativerather than purely summative evaluation, i.e. to assess both ‘whether’ and ‘how’ programme outcomesare achieved. In this paper, we assess the RCT+ approach through an evaluation of Creative Credits – aUK business-to-business innovation voucher initiative intended to promote new innovation partnershipsbetween SMEs and creative service providers. The results suggest the potential value of the RCT+ approachto industrial policy evaluation, and the benefits of mixed methods and longitudinal data collection.

Modelling in the evaluation of a manufacturing strategy

This thesis describes research that has developed the principles of a modelling tool for the analytical evaluation of a manufacturing strategy. The appropriate process of manufacturing strategy formulation is based on mental synthesis with formal planning processes supporting this role. Inherent to such processes is a stage where the effects of alternative strategies on the performance of a manufacturing system must be evaluated so that a choice of preferred strategy can be made. Invariably this evaluation is carried out by practitioners applying mechanisms of judgement, bargaining and analysis. Ibis thesis makes a significant and original contribution to the provision of analytical support for practitioners in this role. The research programme commences by defining the requirements of analytical strategy evaluation from the perspective of practitioners. A broad taxonomy of models has been used to identify a set of potentially suitable techniques for the strategy evaluation task. Then, where possible, unsuitable modelling techniques have been identified on the basis of evidence in the literature and discarded from this set. The remaining modelling techniques have been critically appraised by testing representative contemporary modelling tools in an industrially based experimentation programme. The results show that individual modelling techniques exhibit various limitations in the strategy evaluation role, though some combinations do appear to provide the necessary functionality. On the basis of this comprehensive and in-depth knowledge a modelling tool ' has been specifically designed for this task. Further experimental testing has then been conducted to verify the principles of this modelling tool. Ibis research has bridged the fields of manufacturing strategy formulation and manufacturing systems modelling and makes two contributions to knowledge. Firstly, a comprehensive and in-depth platform of knowledge has been established about modelling techniques in manufacturing strategy evaluation. Secondly, the principles of a tool that supports this role have been formed and verified.

Modelling in the evaluation of a manufacturing strategy

This thesis describes research that has developed the principles of a modelling tool for the analytical evaluation of a manufacturing strategy. The appropriate process of manufacturing strategy formulation is based on mental synthesis with formal planning processes supporting this role. Inherent to such processes is a stage where the effects of alternative strategies on the performance of a manufacturing system must be evaluated so that a choice of preferred strategy can be made. Invariably this evaluation is carried out by practitioners applying mechanisms of judgement, bargaining and analysis. Ibis thesis makes a significant and original contribution to the provision of analytical support for practitioners in this role. The research programme commences by defining the requirements of analytical strategy evaluation from the perspective of practitioners. A broad taxonomy of models has been used to identify a set of potentially suitable techniques for the strategy evaluation task. Then, where possible, unsuitable modelling techniques have been identified on the basis of evidence in the literature and discarded from this set. The remaining modelling techniques have been critically appraised by testing representative contemporary modelling tools in an industrially based experimentation programme. The results show that individual modelling techniques exhibit various limitations in the strategy evaluation role, though some combinations do appear to provide the necessary functionality. On the basis of this comprehensive and in-depth knowledge a modelling tool ' has been specifically designed for this task. Further experimental testing has then been conducted to verify the principles of this modelling tool. Ibis research has bridged the fields of manufacturing strategy formulation and manufacturing systems modelling and makes two contributions to knowledge. Firstly, a comprehensive and in-depth platform of knowledge has been established about modelling techniques in manufacturing strategy evaluation. Secondly, the principles of a tool that supports this role have been formed and verified.

Developing a robust self evaluation framework for active learning:the first stage of an ERASMUS+ project (QAEMarketPlace4HEI)

In ensuring the quality of learning and teaching in Higher Education, self-evaluation is an important component of the process. An example would be the approach taken within the CDIO community whereby self-evaluation against the CDIO standards is part of the quality assurance process. Eight European universities (Reykjavik University, Iceland; Turku University of Applied Sciences, Finland; Aarhus University, Denmark; Helsinki Metropolia University of Applied Sciences, Finland; Ume? University, Sweden; Telecom Bretagne, France; Aston University, United Kingdom; Queens University Belfast, United Kingdom) are engaged in an EU funded Erasmus + project that is exploring the quality assurance process associated with active learning. The development of a new self-evaluation framework that feeds into a ?Marketplace? where participating institutions can be paired up and then engage in peer evaluations and sharing around each institutions approach to and implementation of active learning. All of the partner institutions are engaged in the application of CDIO within their engineering programmes and this has provided a common starting point for the partnership to form and the project to be developed. Although the initial focus will be CDIO, the longer term aim is that the approach could be of value beyond CDIO and within other disciplines. The focus of this paper is the process by which the self-evaluation framework is being developed and the form of the draft framework. In today?s Higher Education environment, the need to comply with Quality Assurance standards is an ever present feature of programme development and review. When engaging in a project that spans several countries, the wealth of applicable standards and guidelines is significant. In working towards the development of a robust Self Evaluation Framework for this project, the project team decided to take a wide view of the available resources to ensure a full consideration of different requirements and practices. The approach to developing the framework considered: a) institutional standards and processes b) national standards and processes e.g. QAA in the UK c) documents relating to regional / global accreditation schemes e.g. ABET d) requirements / guidelines relating to particular learning and teaching frameworks e.g. CDIO. The resulting draft self-evaluation framework is to be implemented within the project team to start with to support the initial ?Marketplace? pairing process. Following this initial work, changes will be considered before a final version is made available as part of the project outputs. Particular consideration has been paid to the extent of the framework, as a key objective of the project is to ensure that the approach to quality assurance has impact but is not overly demanding in terms of time or paperwork. In other words that it is focused on action and value added to staff, students and the programmes being considered.

GPs' perspectives on managing the NHS Health Check in primary care:a qualitative evaluation of implementation in one area of England

OBJECTIVES: To evaluate the implementation of the National Health Service (NHS) Health Check programme in one area of England from the perspective of general practitioners (GPs). DESIGN: A qualitative exploratory study was conducted with GPs and other healthcare professionals involved in delivering the NHS Health Check and with patients. This paper reports the experience of GPs and focuses on the management of the Heath Check programme in primary care. SETTING: Primary care surgeries in the Heart of Birmingham region (now under the auspices of the Birmingham Cross City Clinical Commissioning Group) were invited to take part in the larger scale evaluation. This study focuses on a subset of those surgeries whose GPs were willing to participate. PARTICIPANTS: 9 GPs from different practices volunteered. GPs served an ethnically diverse region with areas of socioeconomic deprivation. Ethnicities of participant GPs included South Asian, South Asian British, white, black British and Chinese. METHODS: Individual semistructured interviews were conducted with GPs face to face or via telephone. Thematic analysis was used to analyse verbatim transcripts. RESULTS: Themes were generated which represent GPs' experiences of managing the NHS Health Check: primary care as a commercial enterprise; 'buy in' to concordance in preventive healthcare; following protocol and support provision. These themes represent the key issues raised by GPs. They reveal variability in the implementation of NHS Health Checks. GPs also need support in allocating resources to the Health Check including training on how to conduct checks in a concordant (or collaborative) way. CONCLUSIONS: The variability observed in this small-scale evaluation corroborates existing findings suggesting a need for more standardisation. Further large-scale research is needed to determine how that could be achieved. Work needs to be done to further develop a concordant approach to lifestyle advice which involves tailored individual goal setting rather than a paternalistic advice-giving model.

Important factors in predicting mortality outcome from stroke : findings from the Anglia Stroke Clinical Network Evaluation Study

The Anglia Stroke Clinical Network Evaluation Study (ASCNES) is funded by the National Institute for Health Research (NIHR) Research for Patient Benefit Programme (PB-PG-1208-18240). This paper presents independent research funded by the NIHR under its Research for Patient Benefit (RfPB) programme (Grant Reference Number PB-PG-1208-18240). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. EAW receives funding support from the NIHR Biomedical Research Centre award to Cambridge

Costs, quality of life and cost-effectiveness of open and arthroscopic repair for rotator cuff tears : economic evaluation alongside the UKUFF trial

FUNDING & ACKNOWLEDGEMENTS This project was funded by the NIHR Health Technology Assessment programme (project number 05/47/02) and is published in full in Health Technology Assessment; Vol. 19, No. 80. Further information available at: http://www.nets.nihr.ac.uk/projects/hta/054702 This paper presents independent research commissioned by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, MRC, CCF, NETSCC, the HTA programme or the Department of Health. Due to the confidential nature of the trial data supporting this publication not all of the data can be made accessible to other researchers. Please contact the UKUFF study principal investigator Andrew Carr (andrew.carr@ndorms.ox.ac.uk) for more information. The authors wish to thank the UKUFF trial collaborators for their contribution in managing the conduct of the trial, and for their comments on the interim economic results: Marion Campbell and Hannah Bruhn (Centre for Healthcare Randomised Trials, HSRU, University of Aberdeen), Jonathan Rees MD and David Beard (NDORMS, University of Oxford; NIHR Oxford Biomedical Research Centre), Jane Moser (NDORMS, University of Oxford), Raymond Fitzpatrick and Jill Dawson (NDPH, University of Oxford).

Évaluation d’un programme de formation en matière d’agression sexuelle envers les enfants pour les communautés autochtones du Québec

L’agression sexuelle (AS) envers les enfants est une problématique majeure affectant les communautés autochtones. Considérant les conséquences à long terme associées à l’AS, il est primordial que les intervenants soient formés adéquatement pour intervenir en cas d’AS puisque que cette problématique compromet le bien-être des enfants, de leur famille et de leur communauté. Le Centre d’expertise Marie-Vincent, en partenariat avec les communautés autochtones, a mis sur pied une formation en matière d’AS spécifiquement adaptée aux besoins des intervenants. De 2013 à 2015, 161 professionnels œuvrant dans les communautés autochtones ont participé à cette formation. Deux phases de collecte de données distinctes ont été menées pour vérifier l’atteinte des objectifs du programme, soit l’amélioration des connaissances et des interventions en matière d’AS des participants. Lors de la première phase, les participants ont répondu à un questionnaire de réactions et à un test de connaissances. Une étude de cas multiples qualitative comportant quatre communautés a été réalisée lors de la deuxième phase. En plus d’un taux d’appréciation élevé, les résultats révèlent une augmentation significative des connaissances, passant de 54 % à 87 % (t(125) = 16,4, p < 0,001, ηp2 = 0,68). L’analyse des entrevues fait état d’une appropriation des contenus de formation dans la pratique professionnelle des participants. Ces derniers affirment avoir modifié la façon dont ils questionnent les enfants et être plus attentifs aux signes et symptômes d’une AS. Des changements sur le plan des attitudes, notamment sur l’importance du signalement, sont également rapportés. Finalement, la formation est considérée comme l’un des éléments déclencheurs d’une meilleure collaboration intersectorielle dans au moins une communauté.

Enhancing public speaking skills - an evaluation of the Presentation Trainer in the wild

Full paper presented at EC-TEL 2016