Performance Information and Innovation in the Canadian Government (Working Paper 43)

The presentation made at the conference addressed the issue of linkages between performance information and innovation within the Canadian federal government1. This is a three‐part paper prepared as background to that presentation. • Part I provides an overview of three main sources of performance information - results-based systems, program evaluation, and centrally driven review exercises – and reviews the Canadian experience with them. • Part II identifies and discusses a number of innovation issues that are common to the literature reviewed for this paper. • Part III examines actual and potential linkages between innovation and performance information. This section suggests that innovation in the Canadian federal government tends to cluster into two groups: smaller initiatives driven by staff or middle management; and much larger projects involving major programs, whole departments or whole-of-government. Readily available data on smaller innovation projects is skimpy but suggests that performance information does not play a major role in stimulating these initiatives. In contrast, two of the examples of large-scale innovation show that performance information plays a critical role at all stages. The paper concludes by supporting the contention of others writing on this topic: that more research is needed on innovation, particularly on its link to performance information. In that context, other conclusions drawn in this paper are tentative but suggest that the quality of performance information is as important for innovation as it is for performance management. However, innovation is likely to require its own particular performance information that may not be generated on a routine basis for purposes of performance management, particularly in the early stages of innovation. And, while the availability of performance information can be an important success factor in innovation, it does not stand alone. The commonality of a number of other factors identified in the literature surveyed for this paper strongly suggests that equal if not greater priority needs to be given to attenuating factors that inhibit innovation and to nurturing incentives.

IMPROVING THE PERFORMANCE OF DATABASE-CENTRIC APPLICATIONS THROUGH PROGRAM ANALYSIS

Modern software applications are becoming more dependent on database management systems (DBMSs). DBMSs are usually used as black boxes by software developers. For example, Object-Relational Mapping (ORM) is one of the most popular database abstraction approaches that developers use nowadays. Using ORM, objects in Object-Oriented languages are mapped to records in the database, and object manipulations are automatically translated to SQL queries. As a result of such conceptual abstraction, developers do not need deep knowledge of databases; however, all too often this abstraction leads to inefficient and incorrect database access code. Thus, this thesis proposes a series of approaches to improve the performance of database-centric software applications that are implemented using ORM. Our approaches focus on troubleshooting and detecting inefficient (i.e., performance problems) database accesses in the source code, and we rank the detected problems based on their severity. We first conduct an empirical study on the maintenance of ORM code in both open source and industrial applications. We find that ORM performance-related configurations are rarely tuned in practice, and there is a need for tools that can help improve/tune the performance of ORM-based applications. Thus, we propose approaches along two dimensions to help developers improve the performance of ORM-based applications: 1) helping developers write more performant ORM code; and 2) helping developers configure ORM configurations. To provide tooling support to developers, we first propose static analysis approaches to detect performance anti-patterns in the source code. We automatically rank the detected anti-pattern instances according to their performance impacts. Our study finds that by resolving the detected anti-patterns, the application performance can be improved by 34% on average. We then discuss our experience and lessons learned when integrating our anti-pattern detection tool into industrial practice. We hope our experience can help improve the industrial adoption of future research tools. However, as static analysis approaches are prone to false positives and lack runtime information, we also propose dynamic analysis approaches to further help developers improve the performance of their database access code. We propose automated approaches to detect redundant data access anti-patterns in the database access code, and our study finds that resolving such redundant data access anti-patterns can improve application performance by an average of 17%. Finally, we propose an automated approach to tune performance-related ORM configurations using both static and dynamic analysis. Our study shows that our approach can help improve application throughput by 27--138%. Through our case studies on real-world applications, we show that all of our proposed approaches can provide valuable support to developers and help improve application performance significantly.