Performance measurement and benchmarking of a major innovation programme

Purpose: The purpose of this paper is to describe the problems encountered and the solutions developed when using benchmarking and key performance indicators (KPIs) to monitor a major UK social house building innovation (change) programme. The innovation programme sought improvements to both the quality of the house product and the procurement process. Design/methodology/approach: Benchmarking and KPIs were used to quantify performance and in-depth case studies to identify underlying cause and effect relationships within the innovation programme. Findings: The inherent competition between consortium members; the complexity of the relationship between the consortium and its strategic partner; the lack of an authoritative management control structure; and the rapidly changing nature of the UK social housing market all proved problematic to the development of a reliable and robust monitoring system. These problems were overcome by the development of multi-dimensional benchmarking model that balanced the needs and aspirations of the individual organisations with the broader objectives of the consortium. Research limitations/implications: Whilst the research methodology provides insight into the factors that affected the performance of a major innovation programme its findings may not be representative of all projects. Practical implications: The lessons learnt should assist those developing benchmarking models for multi-client consortia. Originality/value: The work reported in this paper describes an inclusive approach to benchmarking in which a multiple client group and their strategic partner sought to work together for shared gain. Very few papers have addressed this issue.

Across the bridge: CLEF 2001 - Non-English monolingual retrieval. The French task.

This paper presents work on document retrieval based on first time participation in the CLEF 2001 monolingual retrieval task using French. The experiment findings indicated that Okapi, the text retrieval system in use, can successfully be used for non-English text retrieval. A lot of internal pre-processing is required in the basic search system for conversion into Okapi access formats. Various shell scripts were written to achieve the conversion in a UNIX environment, failure of which would significantly have impeded the overall performance. Based on the experiment findings using Okapi - originally designed for English - it was clear that, although most European languages share conventional word boundaries and variant word morphemes formed by the additon of suffixes, there is significant difference between French and English retrieval depending on the adaptation of indexing and search strategies in use. No sophisticated method for higher recall and precision such as stemming techniques, phrase translation or de-compounding was employed for the experiment and our results were suggestively poor. Future participation would include more refined query translation tools.

Flexible self-adjustment of task deployment in dynamic environments

This paper presents an investigation into dynamic self-adjustment of task deployment and other aspects of self-management, through the embedding of multiple policies. Non-dedicated loosely-coupled computing environments, such as clusters and grids are increasingly popular platforms for parallel processing. These abundant systems are highly dynamic environments in which many sources of variability affect the run-time efficiency of tasks. The dynamism is exacerbated by the incorporation of mobile devices and wireless communication. This paper proposes an adaptive strategy for the flexible run-time deployment of tasks; to continuously maintain efficiency despite the environmental variability. The strategy centres on policy-based scheduling which is informed by contextual and environmental inputs such as variance in the round-trip communication time between a client and its workers and the effective processing performance of each worker. A self-management framework has been implemented for evaluation purposes. The framework integrates several policy-controlled, adaptive services with the application code, enabling the run-time behaviour to be adapted to contextual and environmental conditions. Using this framework, an exemplar self-managing parallel application is implemented and used to investigate the extent of the benefits of the strategy