R&D Performance Analysis: Case Studies on the Challenges and Promotion of the Evaluation and Measurement of R&D

Due to the intense international competition, demanding, and sophisticated customers, and diverse transforming technological change, organizations need to renew their products and services by allocating resources on research and development (R&D). Managing R&D is complex, but vital for many organizations to survive in the dynamic, turbulent environment. Thus, the increased interest among decision-makers towards finding the right performance measures for R&D is understandable. The measures or evaluation methods of R&D performance can be utilized for multiple purposes; for strategic control, for justifying the existence of R&D, for providing information and improving activities, as well as for the purposes of motivating and benchmarking. The earlier research in the field of R&D performance analysis has generally focused on either the activities and considerable factors and dimensions - e.g. strategic perspectives, purposes of measurement, levels of analysis, types of R&D or phases of R&D process - prior to the selection of R&Dperformance measures, or on proposed principles or actual implementation of theselection or design processes of R&D performance measures or measurement systems. This study aims at integrating the consideration of essential factors anddimensions of R&D performance analysis to developed selection processes of R&D measures, which have been applied in real-world organizations. The earlier models for corporate performance measurement that can be found in the literature, are to some extent adaptable also to the development of measurement systemsand selecting the measures in R&D activities. However, it is necessary to emphasize the special aspects related to the measurement of R&D performance in a way that make the development of new approaches for especially R&D performance measure selection necessary: First, the special characteristics of R&D - such as the long time lag between the inputs and outcomes, as well as the overall complexity and difficult coordination of activities - influence the R&D performance analysis problems, such as the need for more systematic, objective, balanced and multi-dimensional approaches for R&D measure selection, as well as the incompatibility of R&D measurement systems to other corporate measurement systems and vice versa. Secondly, the above-mentioned characteristics and challenges bring forth the significance of the influencing factors and dimensions that need to be recognized in order to derive the selection criteria for measures and choose the right R&D metrics, which is the most crucial step in the measurement system development process. The main purpose of this study is to support the management and control of the research and development activities of organizations by increasing the understanding of R&D performance analysis, clarifying the main factors related to the selection of R&D measures and by providing novel types of approaches and methods for systematizing the whole strategy- and business-based selection and development process of R&D indicators.The final aim of the research is to support the management in their decision making of R&D with suitable, systematically chosen measures or evaluation methods of R&D performance. Thus, the emphasis in most sub-areas of the present research has been on the promotion of the selection and development process of R&D indicators with the help of the different tools and decision support systems, i.e. the research has normative features through providing guidelines by novel types of approaches. The gathering of data and conducting case studies in metal and electronic industry companies, in the information and communications technology (ICT) sector, and in non-profit organizations helped us to formulate a comprehensive picture of the main challenges of R&D performance analysis in different organizations, which is essential, as recognition of the most importantproblem areas is a very crucial element in the constructive research approach utilized in this study. Multiple practical benefits regarding the defined problemareas could be found in the various constructed approaches presented in this dissertation: 1) the selection of R&D measures became more systematic when compared to the empirical analysis, as it was common that there were no systematic approaches utilized in the studied organizations earlier; 2) the evaluation methods or measures of R&D chosen with the help of the developed approaches can be more directly utilized in the decision-making, because of the thorough consideration of the purpose of measurement, as well as other dimensions of measurement; 3) more balance to the set of R&D measures was desired and gained throughthe holistic approaches to the selection processes; and 4) more objectivity wasgained through organizing the selection processes, as the earlier systems were considered subjective in many organizations. Scientifically, this dissertation aims to make a contribution to the present body of knowledge of R&D performance analysis by facilitating dealing with the versatility and challenges of R&D performance analysis, as well as the factors and dimensions influencing the selection of R&D performance measures, and by integrating these aspects to the developed novel types of approaches, methods and tools in the selection processes of R&D measures, applied in real-world organizations. In the whole research, facilitation of dealing with the versatility and challenges in R&D performance analysis, as well as the factors and dimensions influencing the R&D performance measure selection are strongly integrated with the constructed approaches. Thus, the research meets the above-mentioned purposes and objectives of the dissertation from the scientific as well as from the practical point of view.

Exploration des conceptions de la performance privilégiées par des infirmières et des membres de l’équipe d’encadrement impliqués dans l’offre de services infirmiers : une étude qualitative exploratoire

Problématique : Les organisations de santé font face à des pressions diverses pour offrir des soins et des services qui répondent aux plus hauts critères de performance et rendre compte de cette performance. Ces pressions proviennent de différents acteurs tels que les usagers du système de santé et les décideurs politiques. En raison de la place importante que prennent les infirmières dans l’offre de services de santé, il existe un intérêt croissant pour la mise en place d’interventions visant à mesurer et à améliorer la performance des services infirmiers. Cependant, dans le cadre de ces processus, les organisations sont souvent confrontées à des visions différentes et conflictuelles de la performance et à diverses approches pour la mesurer. Objectifs : Cette étude qualitative exploratoire a pour but d’explorer les conceptions de la performance des membres de l’équipe d’encadrement impliqués dans la prestation des services infirmiers et des infirmières soignantes et d’examiner dans quelle mesure les conceptions de la performance des deux groupes d’acteurs correspondent ou entrent en conflit. Méthodologie : Des entrevues semi-dirigées ont été conduites auprès de cinq membres de l’équipe d’encadrement et de trois infirmières. Une analyse de contenu a été effectuée à la fois pour faire ressortir l’éventail des conceptions et celles qui sont les plus prépondérantes dans les discours. Le cadre de référence ayant guidé cette analyse est une adaptation du modèle conceptuel de Donabedian comprenant trois dimensions soit la structure, le processus et les résultats (Unruh & Wan, 2004). Résultats : L’analyse des données recueillies auprès des membres de l’équipe d’encadrement fait ressortir dix conceptions distinctes, mais interreliées de la performance qui mettent l’accent sur les éléments de processus de soins infirmiers et de résultats chez la clientèle. Concernant les infirmières, neuf conceptions ont été répertoriées et l’accent a été porté surtout sur les éléments concernant l’adéquation des ressources humaines infirmières et de processus de soins infirmiers. Certaines similitudes et différences ont été repérées entre les conceptions de ces deux groupes d’acteurs. Conclusion : Cette étude permet de mieux comprendre les conceptions de la performance des acteurs impliqués dans l’offre de services infirmiers. Le modèle intégrateur qui résulte de la combinaison de ces différentes conceptions offre un cadre utile pour guider la construction d’outils de mesure de performance directement en lien avec les soins infirmiers et répondre à la demande d’imputabilité par rapport à ces services.

Indicateurs de la performance des soins infirmiers ambulatoires : une scoping review

Travail dirigé présenté à la Faculté des sciences infirmières en vue de l’obtention du grade de maîtrise en sciences infirmières option administration des services infirmiers

Implanter et pérenniser un système de gestion de la performance des services infirmiers

Travail dirigé présenté à la Faculté des sciences infirmières en vue de l’obtention du grade de Maître ès sciences (M.Sc.) en sciences infirmières option administration en sciences infirmières

Organizational excellence through quality governance

This paper provides an overview of an Enterprise Engineering method called IEM (Integrated Enterprise Modeling) and it describes the usage of SIX SIGMA approach for performance measurement of an enterprise. Based on these two instruments a methodology including procedure and tools is developed, which allows enterprises to define their adequate quality criteria for performance, to measure performance and quality and to derive reasonable actions to take for optimization. Determination of adequate quality and performance criteria and reasonable measures for the enterprise organization means “Quality Governance”.

Quality certification, performance and size in hotel chains

This paper examines the relationship between quality certification and performance, and quality certification and size in hotel chains operating in Spain. In an initial phase, a quantitative study is made with secondary and objective data to analyse these relationships. In a second phase, a qualitative analysis is applied to reach a better understanding of the quantitative results. The findings show that chains with certified hotels achieve better performance levels; that better performance levels increase with the percentage of certified hotels within the chain; and that quality certification has positive effects on some performance variables. In addition, size is not a key factor for certification, although it could be an enabler.

Mesure de la performance dans les organisations de réadaptation en déficience physique

La présente thèse vise à évaluer le degré d’implantation et d’utilisation de systèmes de mesure de la performance (SMP) par les décideurs des organisations de réadaptation et à comprendre les facteurs contextuels ayant influencé leur implantation. Pour ce faire, une étude de cas multiples a été réalisée comprenant deux sources de données: des entrevues individuelles avec des cadres supérieurs des organisations de réadaptation du Québec et des documents organisationnels. Le cadre conceptuel Consolidated Framework for Implementation Research a été utilisé pour guider la collecte et l’analyse des données. Une analyse intra-cas ainsi qu’une analyse inter-cas ont été réalisées. Nos résultats montrent que le niveau de préparation organisationnelle à l’implantation d’un SMP était élevé et que les SMP ont été implantés avec succès et utilisés de plusieurs façons. Les organisations les ont utilisés de façon passive (comme outil d’information), de façon ciblée (pour tenter d’améliorer des domaines sous-performants) et de façon politique (comme outil de négociation auprès des autorités gouvernementales). Cette utilisation diversifiée des SMP est suscitée par l’interaction complexe de facteurs provenant du contexte interne propre à chaque organisation, des caractéristiques du SMP, du processus d’implantation appliqué et du contexte externe dans lequel évoluent ces organisations. Au niveau du contexte interne, l’engagement continu et le leadership de la haute direction ont été décisifs dans l’implantation du SMP de par leur influence sur l’identification du besoin d’un SMP, l’engagement des utilisateurs visés dans le projet, la priorité organisationnelle accordée au SMP ainsi que les ressources octroyées à son implantation, la qualité des communications et le climat d’apprentissage organisationnel. Toutefois, même si certains de ces facteurs, comme les ressources octroyées à l’implantation, la priorité organisationnelle du SMP et le climat d’apprentissage se sont révélés être des barrières à l’implantation, ultimement, ces barrières n’étaient pas suffisamment importantes pour entraver l’utilisation du SMP. Cette étude a également confirmé l’importance des caractéristiques du SMP, particulièrement la perception de qualité et d’utilité de l’information. Cependant, à elles seules, ces caractéristiques sont insuffisantes pour assurer le succès d’implantation. Cette analyse d’implantation a également révélé que, même si le processus d’implantation ne suit pas des étapes formelles, un plan de développement du SMP, la participation et l’engagement des décideurs ainsi que la désignation d’un responsable de projet ont tous facilité son implantation. Cependant, l’absence d’évaluation et de réflexion collective sur le processus d’implantation a limité le potentiel d’apprentissage organisationnel, un prérequis à l’amélioration de la performance. Quant au contexte externe, le soutien d’un organisme externe s’est avéré un facilitateur indispensable pour favoriser l’implantation de SMP par les organisations de réadaptation malgré l’absence de politiques et incitatifs gouvernementaux à cet effet. Cette étude contribue à accroître les connaissances sur les facteurs contextuels ainsi que sur leurs interactions dans l’utilisation d’innovations tels les SMP et confirme l’importance d’aborder l’analyse de l’implantation avec une perspective systémique.

Understanding and choosing clinical performance measures for quality improvement : development of a typology : final report /

"Contract no. 282-92-0038, delivery order #3."

Using clinical indicators in a quality improvement programme targeting cardiac care

Rationale. The Brisbane Cardiac Consortium, a quality improvement collaboration of clinicians from three hospitals and five divisions of general practice, developed and reported clinical indicators as measures of the quality of care received by patients with acute coronary syndromes or congestive heart failure. Development of indicators. An expert panel derived indicators that measured gaps between evidence and practice. Data collected from hospital records and general practice heart-check forms were used to calculate process and outcome indicators for each condition. Our indicators were reliable (kappa scores 0.7-1.0) and widely accepted by clinicians as having face validity. Independent review of indicator-failed, in-hospital cases revealed that, for 27 of 28 process indicators, clinically legitimate reasons for withholding specific interventions were found in

Application of analytic hierarchy process for measuring and comparing the global performance of intensive care units

Purpose: To develop a model for the global performance measurement of intensive care units (ICUs) and to apply that model to compare the services for quality improvement. Materials and Methods: Analytic hierarchy process, a multiple-attribute decision-making technique, is used in this study to evolve such a model. The steps consisted of identifying the critical success factors for the best performance of an ICU, identifying subfactors that influence the critical factors, comparing them pairwise, deriving their relative importance and ratings, and calculating the cumulative performance according to the attributes of a given ICU. Every step in the model was derived by group discussions, brainstorming, and consensus among intensivists. Results: The model was applied to 3 ICUs, 1 each in Barbados, Trinidad, and India in tertiary care teaching hospitals of similar setting. The cumulative performance rating of the Barbados ICU was 1.17 when compared with that of Trinidad and Indian ICU, which were 0.82 and 0.75, respectively, showing that the Trinidad and Indian ICUs performed 70% and 64% with respect to Barbados ICU. The model also enabled identifying specific areas where the ICUs did not perform well, which helped to improvise those areas. Conclusions: Analytic hierarchy process is a very useful model to measure the global performance of an ICU. © 2005 Elsevier Inc. All rights reserved.

Managing the quality, value and impact of academic libraries: the role of external standards

Purpose – The purpose of this paper is to explore the role and relevance of external standards in demonstrating the value and impact of academic library services to their stakeholders. Design/methodology/approach – Two UK standards, Charter Mark and Customer Service Excellence, are evaluated via an exploratory case study, employing multiple data collection techniques. Methods and results of phases 1-2 of a three phase research project are outlined. Findings – Despite some limitations, standards may assist the manager in demonstrating the value, impact and quality of academic libraries in a recessional environment. Active engagement and partnership with customers is imperative if academic libraries are to be viewed as vital to their parent organisations and thus survive. Originality/value – This paper provides a systematic evaluation of the role of external accreditation standards in measuring academic library service value and impact.

Emergent strategy and the measurement of performance:the formulation of performance indicators at the microlevel

This article presents some evidence on an aspect of the design of a strategic control system, at the microlevel, within a single organization. The research we report used an ethnographic approach to provide an understanding of strategy formulation. Our aim is to contribute to an area of literature which is of increasing significance, but relatively underdeveloped in terms of the application of in-depth, field-research techniques. We take an intensive look at the manner in which performance measures are formulated, at the microlevel, within a single organization. The article presents, as an in-depth case analysis, the experience of a fisheries holding company in New Zealand. The article recounts the experiences of managers within the organization of the process of identification of such things as critical success factors and key performance indicators (KPIs) and, more broadly, the formulation of a strategic performance measurement system.

Improving performance? The adoption and implementation of quality systems in U.K. nonprofits

There are increasing and multiple pressures on nonprofit organizations to demonstrate excellence in performance. Although there is a growing literature on the various approaches to performance improvement taken by nonprofits, little is known about the processes involved in the adoption and implementation of specific approaches. This article is about the adoption and use of one approach to performance improvement, quality systems, in the U.K. nonprofit sector. We report findings about factors that encourage nonprofits to adopt quality systems. We also analyze the distinctive challenges of implementing quality approaches in a nonprofit sector context and suggest critical success factors. The article concludes with a discussion of the organizational and policy implications of applying the management concepts of quality and performance to the nonprofit sector.

Measurement of hospital performance

The rationale for carrying out this research was to address the clear lack of knowledge surrounding the measurement of public hospital performance in Ireland. The objectives of this research were to develop a comprehensive model for measuring hospital performance and using this model to measure the performance of public acute hospitals in Ireland in 2007. Having assessed the advantages and disadvantages of various measurement models the Data Envelopment Analysis (DEA) model was chosen for this research. DEA was initiated by Charnes, Cooper and Rhodes in 1978 and further developed by Fare et al. (1983) and Banker et al. (1984). The method used to choose relevant inputs and outputs to be included in the model followed that adopted by Casu et al. (2005) which included the use of focus groups. The main conclusions of the research are threefold. Firstly, it is clear that each stakeholder group has differing opinions on what constitutes good performance. It is therefore imperative that any performance measurement model would be designed within parameters that are clearly understood by any intended audience. Secondly, there is a lack of publicly available qualitative information in Ireland that inhibits detailed analysis of hospital performance. Thirdly, based on available qualitative and quantitative data the results indicated a high level of efficiency among the public acute hospitals in Ireland in their staffing and non pay costs, averaging 98.5%. As DEA scores are sensitive to the number of input and output variables as well as the size of the sample it should be borne in mind that a high level of efficiency could be as a result of using DEA with too many variables compared to the number of hospitals. No hospital was deemed to be scale efficient in any of the models even though the average scale efficiency for all of the hospitals was relatively high at 90.3%. Arising from this research the main recommendations would be that information on medical outcomes, survival rates and patient satisfaction should be made publicly available in Ireland; that despite a high average efficiency level that many individual hospitals need to focus on improving their technical and scale efficiencies, and that performance measurement models should be developed that would include more qualitative data.

General and multiplicative non-parametric corporate performance models with interval ratio data

The increasing intensity of global competition has led organizations to utilize various types of performance measurement tools for improving the quality of their products and services. Data envelopment analysis (DEA) is a methodology for evaluating and measuring the relative efficiencies of a set of decision making units (DMUs) that use multiple inputs to produce multiple outputs. All the data in the conventional DEA with input and/or output ratios assumes the form of crisp numbers. However, the observed values of data in real-world problems are sometimes expressed as interval ratios. In this paper, we propose two new models: general and multiplicative non-parametric ratio models for DEA problems with interval data. The contributions of this paper are fourfold: (1) we consider input and output data expressed as interval ratios in DEA; (2) we address the gap in DEA literature for problems not suitable or difficult to model with crisp values; (3) we propose two new DEA models for evaluating the relative efficiencies of DMUs with interval ratios, and (4) we present a case study involving 20 banks with three interval ratios to demonstrate the applicability and efficacy of the proposed models where the traditional indicators are mostly financial ratios. © 2011 Elsevier Inc.