Beyond words: Testing alignment among inter-organizational performance measures

Purpose: This paper seeks to measure in a quantitative way the degree of alignment among a set of performance measures between two organizations. Design/methodology/approach: This paper extends Venkatraman's test of coalignment to assess the alignment of a set of performance measures governing a contractual inter-organizational relationship. The authors applied the test and present coefficients of misalignment across three sets of measures: those used by a service provider involved in the research, those used by customers contracting the services, and those documented in 11 contracts studied. Findings: Results confirmed a high degree of alignment between target and actual operational performance in the contracts. The alignment of customers' financial objectives and contracts' operational metrics was low. Calculations show poor alignment between the objectives of the provider and the contribution received from the contracts. Research limitations/implications: Some limitations of the conclusions include the small sample of contracts used in the calculations. Further research should include not only actual contracts, but also failed ones. Practical implications: It is possible that misaligned goals, represented in misaligned performance measures, lead to tensions in intra-firm relationships. If these tensions are not addressed properly the relationship could be unstable or terminated prematurely. This method of measuring alignment could detect early potential dangers in intra-firm relationships. Originality/value: This paper extends Venkatraman's test of coalignment to assess the alignment of a set of performance measures governing a contractual inter-organizational relationship. Management researchers and business professionals may use this methodology when exploring degrees of alignment of performance measures in intra-functional and inter-firm relationships. © Emerald Group Publishing Limited.

Uncertainty management for on-line optimisation of a POMDP-based large-scale spoken dialogue system

The optimization of dialogue policies using reinforcement learning (RL) is now an accepted part of the state of the art in spoken dialogue systems (SDS). Yet, it is still the case that the commonly used training algorithms for SDS require a large number of dialogues and hence most systems still rely on artificial data generated by a user simulator. Optimization is therefore performed off-line before releasing the system to real users. Gaussian Processes (GP) for RL have recently been applied to dialogue systems. One advantage of GP is that they compute an explicit measure of uncertainty in the value function estimates computed during learning. In this paper, a class of novel learning strategies is described which use uncertainty to control exploration on-line. Comparisons between several exploration schemes show that significant improvements to learning speed can be obtained and that rapid and safe online optimisation is possible, even on a complex task. Copyright © 2011 ISCA.

Is performance measurement and management fit for the future?

Performance measurement and management (PMM) is a management and research paradox. On one hand, it provides management with many critical, useful, and needed functions. Yet, there is evidence that it can adversely affect performance. This paper attempts to resolve this paradox by focusing on the issue of "fit". That is, in today's dynamic and turbulent environment, changes in either the business environment or the business strategy can lead to the need for new or revised measures and metrics. Yet, if these measures and metrics are either not revised or incorrectly revised, then we can encounter situations where what the firm wants to achieve (as communicated by its strategy) and what the firm measures and rewards are not synchronised with each other (i.e., there is a lack of "fit"). This situation can adversely affect the ability of the firm to compete. The issue of fit is explored using a three phase Delphi approach. Initially intended to resolve this first paradox, the Delphi study identified another paradox - one in which the researchers found that in a dynamic environment, firms do revise their strategies, yet, often the PMM system is not changed. To resolve this second paradox, the paper proposes a new framework - one that shows that under certain conditions, the observed metrics "lag" is not only explainable but also desirable. The findings suggest a need to recast the accepted relationship between strategy and PMM system and the output included the Performance Alignment Matrix that had utility for managers. © 2013 .