Software quality assurance : in large scale and complex software-intensive systems

In recent decades we have seen enormous increases in the capabilities of software intensive systems, resulting in exponential growth in their size and complexity. Software and systems engineers routinely develop systems with advanced functionalities that would not even have been conceived of 20 years ago. This observation was highlighted in the Critical Code report commissioned by the US Department of Defense in 2010, which identified a critical software engineering challenge as theability to deliver “software assurance in the presence of...architectural innovation and complexity, criticality with respect to safety, (and) overall complexity and scale”.

Knowledge mediation in software quality engineering

The risk of failure of the software development process remains high despite many attempts to improve the quality of software engineering. Contemporary approaches to process assurance, such as the capability maturity model have not prevented systemic failures, nor have project management methodologies provided guarantees of software quality. The paper proposes an approach to software quality assurance based on a knowledge mediated concurrent audit, which incorporates essential feedback processes. Through a tightly integrated approach to quality audit, programmers would be empowered to use any chosen methodology to advantage, supported by intelligent monitoring of the essential interactions which occur in the development process. An experimental application implementing some aspects of the proposal is described

Quality Assurance in Extreme Programming

Our previous research about possible quality improvements in Extreme Programming (XP) led us to a conclusion that XP supports many good engineering practices but there is still place for refinements. Our proposal was to add dedicated Quality Assurance (QA) measures, which should be sufficiently effective and at the same time simpler enough in the context of XP. This paper intends to analyze the possibilities for an effective way for applying approved quality assurance practices to XP. The last should not affect negatively to the process and in the meantime must lead to better quality assurance. We aim to make changes to XP that even if would slow down a bit the development process, will make it more suitable for widest range of projects including large and very large projects as well as life critical and highly reliable systems.

A tool to include gamma analysis software into a quality assurance program

PURPOSE: To provide a tool to enable gamma analysis software algorithms to be included in a quality assurance (QA) program.

METHODS: Four image sets were created comprising two geometric images to independently test the distance to agreement (DTA) and dose difference (DD) elements of the gamma algorithm, a clinical step and shoot IMRT field and a clinical VMAT arc. The images were analysed using global and local gamma analysis with 2 in-house and 8 commercially available software encompassing 15 software versions. The effect of image resolution on gamma pass rates was also investigated.

RESULTS: All but one software accurately calculated the gamma passing rate for the geometric images. Variation in global gamma passing rates of 1% at 3%/3mm and over 2% at 1%/1mm was measured between software and software versions with analysis of appropriately sampled images.

CONCLUSION: This study provides a suite of test images and the gamma pass rates achieved for a selection of commercially available software. This image suite will enable validation of gamma analysis software within a QA program and provide a frame of reference by which to compare results reported in the literature from various manufacturers and software versions.

Treatment plan complexity metrics for predicting IMRT pre-treatment quality assurance results

The planning of IMRT treatments requires a compromise between dose conformity (complexity) and deliverability. This study investigates established and novel treatment complexity metrics for 122 IMRT beams from prostate treatment plans. The Treatment and Dose Assessor software was used to extract the necessary data from exported treatment plan files and calculate the metrics. For most of the metrics, there was strong overlap between the calculated values for plans that passed and failed their quality assurance (QA) tests. However, statistically significant variation between plans that passed and failed QA measurements was found for the established modulation index and for a novel metric describing the proportion of small apertures in each beam. The ‘small aperture score’ provided threshold values which successfully distinguished deliverable treatment plans from plans that did not pass QA, with a low false negative rate.

Straw Performance Studies and Quality Assurance for the ATLAS Transition Radiation Tracker

The Transition Radiation Tracker (TRT) of the ATLAS experiment at the LHC is part of the Inner Detector. It is designed as a robust and powerful gaseous detector that provides tracking through individual drift-tubes (straws) as well as particle identification via transition radiation (TR) detection. The straw tubes are operated with Xe-CO2-O2 70/27/3, a gas that combines the advantages of efficient TR absorption, a short electron drift time and minimum ageing effects. The modules of the barrel part of the TRT were built in the United States while the end-cap wheels are assembled at two Russian institutes. Acceptance tests of barrel modules and end-cap wheels are performed at CERN before assembly and integration with the Semiconductor Tracker (SCT) and the Pixel Detector. This thesis first describes simulations the TRT straw tube. The argon-based acceptance gas mixture as well as two xenon-based operating gases are examined for its properties. Drift velocities and Townsend coefficients are computed with the help of the program Magboltz and used to study electron drift and multiplication in the straw using the software Garfield. The inclusion of Penning transfers in the avalanche process leads to remarkable agreements with experimental data. A high level of cleanliness in the TRT s acceptance test gas system is indispensable. To monitor gas purity, a small straw tube detector has been constructed and extensively used to study the ageing behaviour of the straw tube in Ar-CO2. A variety of ageing tests are presented and discussed. Acceptance tests for the TRT survey dimensions, wire tension, gas-tightness, high-voltage stability and gas gain uniformity along each individual straw. The thesis gives details on acceptance criteria and measurement methods in the case of the end-cap wheels. Special focus is put on wire tension and straw straightness. The effect of geometrically deformed straws on gas gain and energy resolution is examined in an experimental setup and compared to simulation studies. An overview of the most important results from the end-cap wheels tested up to this point is presented.

Enabling the evolution of J2EE applications through reverse engineering and quality assurance

Enterprise Applications are complex software systems that manipulate much persistent data and interact with the user through a vast and complex user interface. In particular applications written for the Java 2 Platform, Enterprise Edition (J2EE) are composed using various technologies such as Enterprise Java Beans (EJB) or Java Server Pages (JSP) that in turn rely on languages other than Java, such as XML or SQL. In this heterogeneous context applying existing reverse engineering and quality assurance techniques developed for object-oriented systems is not enough. Because those techniques have been created to measure quality or provide information about one aspect of J2EE applications, they cannot properly measure the quality of the entire system. We intend to devise techniques and metrics to measure quality in J2EE applications considering all their aspects and to aid their evolution. Using software visualization we also intend to inspect to structure of J2EE applications and all other aspects that can be investigate through this technique. In order to do that we also need to create a unified meta-model including all elements composing a J2EE application.

Bankable procedures for the technical quality assurance of large scale PV plants

Strict technical quality assurance procedures are essential for PV plant bankability. When large-scale PV plants are concerned, this is typically accomplished in three consecutive phases: an energy yield forecast, that is performed at the beginning of the project and is typically accomplished by means of a simulation exercise performed with dedicated software; a reception test campaign, that is performed at the end of the commissioning and consists of a set of tests for determining the efficiency and the reliability of the PV plant devices; and a performance analysis of the first years of operation, that consists in comparing the real energy production with the one calculated from the recorded operating conditions and taking into account the maintenance records. In the last six years, IES-UPM has offered both indoor and on-site quality control campaigns for more than 60 PV plants, with an accumulated power of more than 300 MW, in close contact with Engineering, Procurement and Construction Contractors and financial entities. This paper presents the lessons learned from such experience.

Quality concerns in large-scale and complex software-intensive systems

Software quality management (SQM) is the collection of all processes that ensure that software products, services, and life cycle process implementations meet organizational software quality objectives and achieve stakeholder satisfaction. SQM comprises three basic subcategories: software quality planning, software quality assurance (SQA), and software quality control and software process improvement. This chapter provides a general overview of the SQA domain and discuss the related concept. A conceptual model for software quality framework is provided together with the current approaches for SQA. The chapter concludes with some of the identified challenges and future challenges regarding SQA.