User dyads in software testing: bypassing the need for expert observers

This paper arises out of a research study into the online help facilities provided in popular software applications such as word processors. Its particular focus is on experimental methods of evaluating the effectiveness and usability of those facilities. Focus groups, questionnaires, and online surveys had already been used in other phases of the study, but it was judged that these approaches would be unsuitable for measuring effectiveness and usability because they are susceptible to respondents' subjectivity. Direct observation of people working on set word-processing tasks was ruled out initially because of a lack of trained observers; it would have taken too long for the investigator to observe a large enough sample by himself. Automatic recording of users' actions was also rejected, as it would have demanded equipment and/or software that was not available and seemed too expensive to acquire. The approach and techniques described here were an attempt to overcome these difficulties by using observers drawn from the same population of students that provided the test subjects; as a by-product, this may also have enhanced the acceptability (and hence possibly the validity) of the experiments by reducing the exam pressure perceived by participants.

Supporting the software testing process through specification animation

Achieving consistency between a specification and its implementation is an important part of software development. In this paper, we present a method for generating passive test oracles that act as self-checking implementations. The implementation is verified using an animation tool to check that the behavior of the implementation matches the behavior of the specification. We discuss how to integrate this method into a framework developed for systematically animating specifications, which means a tester can significantly reduce testing time and effort by reusing work products from the animation. One such work product is a testgraph: a directed graph that partially models the states and transitions of the specification. Testgraphs are used to generate sequences for animation, and during testing, to execute these same sequences on the implementation.

A framework for table driven testing of Java classes

With the advent of object-oriented languages and the portability of Java, the development and use of class libraries has become widespread. Effective class reuse depends on class reliability which in turn depends on thorough testing. This paper describes a class testing approach based on modeling each test case with a tuple and then generating large numbers of tuples to thoroughly cover an input space with many interesting combinations of values. The testing approach is supported by the Roast framework for the testing of Java classes. Roast provides automated tuple generation based on boundary values, unit operations that support driver standardization, and test case templates used for code generation. Roast produces thorough, compact test drivers with low development and maintenance cost. The framework and tool support are illustrated on a number of non-trivial classes, including a graphical user interface policy manager. Quantitative results are presented to substantiate the practicality and effectiveness of the approach. Copyright (C) 2002 John Wiley Sons, Ltd.

Monitoring and isolation of blood dendritic cells from apheresis products in healthy individuals: a platform for cancer immunotherapy

The fundamental role of dendritic cells (DC in initiating and directing the primary immune response is well established. Furthermore, it is now accepted that DC may be useful in new vaccination strategies for preventing certain malignant and infectious diseases. As blood DC (BDC physiology differs from that of the DC homologues generated in vitro from monocyte precursors, it is becoming more relevant to consider BDC for therapeutic interventions. Until recently, protocols for the isolation of BDC were laborious and inefficient; therefore, their use for investigative cancer immunotherapy is not widespread. In this study, we carefully documented BDC counts, yields and subsets during apheresis (Cobe Spectra), the initial and essential procedure in creating a BDC isolation platform for cancer immunotherapy. We established that an automated software package (Version 6,0 AutoPBPC) provides an operator-independent reliable source of motionuclear cells (MNC for BDC preparation. Further, we observed that BDC might be recovered in high yields, often greater than 100% relative to the number of circulating BDC predicted by blood volume. An average of 66 million (range, 17-179) BDC per 10-1 procedure were obtained, largely satisfying the needs for immunization. Higher yields were possible on total processed blood volumes of 151. BDC were not activated by the isolation procedure and, more importantly, both BDC subsets (CD11c(+)CD123(low) and CD11c(-)CD123(high)) were equally represented. Finally, we established that the apheresis product could be used for antibody-based BDC immunoselection and demonstrated that fully functional BDC can be obtained by this procedure. (C) 2002 Published by Elsevier Science B.V.

Effects of leukapheresis protocol, cell processing and cryopreservation on the generation of monocyte-derived DC for immune therapy

Background Many clinical trials of DC-based immunotherapy involve administration of monocyte-derived DCs (Mo-DC) on multiple occasions. We aimed to determine the optimal cell processing procedures and timing (leukapheresis, RBC depletion and cryopreservation) for generation of Mo-DC for clinical purposes. Methods Leukapheresis was undertaken using a COBE Spectra. Two instrument settings were compared - the standard semi-automated software (Version 4.7) (n = 10) and the fully automated software (Version 6.0) (n = 40). Density gradient centrifugation using Ficoll, Percoll, a combination of these methods or neither for RBC depletion were compared. Outcomes (including cell yield and purity) were compared for cryopreserved unmanipulated monocytes and cryopreserved Mo-DC. Results Software Version 6.0 provided significantly better enrichment for monocytes (P

Formal derivation of finite state machines for class testing

Previous work on generating state machines for the purpose of class testing has not been formally based. There has also been work on deriving state machines from formal specifications for testing non-object-oriented software. We build on this work by presenting a method for deriving a state machine for testing purposes from a formal specification of the class under test. We also show how the resulting state machine can be used as the basis for a test suite developed and executed using an existing framework for class testing. To derive the state machine, we identify the states and possible interactions of the operations of the class under test. The Test Template Framework is used to formally derive the states from the Object-Z specification of the class under test. The transitions of the finite state machine are calculated from the derived states and the class's operations. The formally derived finite state machine is transformed to a ClassBench testgraph, which is used as input to the ClassBench framework to test a C++ implementation of the class. The method is illustrated using a simple bounded queue example.

A framework for specification-based testing

Test templates and a test template framework are introduced as useful concepts in specification-based testing. The framework can be defined using any model-based specification notation and used to derive tests from model-based specifications-in this paper, it is demonstrated using the Z notation. The framework formally defines test data sets and their relation to the operations in a specification and to other test data sets, providing structure to the testing process. Flexibility is preserved, so that many testing strategies can be used. Important application areas of the framework are discussed, including refinement of test data, regression testing, and test oracles.

Automated edge-detection technique for measurement of brachial artery reactivity: A comparison of concordance with manual measurements

Concerns have been raised about the reproducibility of brachial artery reactivity (BAR), because subjective decisions regarding the location of interfaces may influence the measurement of very small changes in lumen diameter. We studied 120 consecutive patients with BAR to address if an automated technique could be applied, and if experience influenced reproducibility between two observers, one experienced and one inexperienced. Digital cineloops were measured automatically, using software that measures the leading edge of the endothelium and tracks this in sequential frames and also manually, where a set of three point-to-point measurements were averaged. There was a high correlation between automated and manual techniques for both observers, although less variability was present with expert readers. The limits of agreement overall for interobserver concordance were 0.13 +/-0.65 mm for the manual and 0.03 +/-0.74 mm for the automated measurement. For intraobserver concordance, the limits of agreement were -0.07 +/-0.38 mm for observer 1 and -0.16 +/-0.55 mm for observer 2. We concluded that BAR measurements were highly concordant between observers, although more concordant using the automated method, and that experience does affect concordance. Care must be taken to ensure that the same segments are measured between observers and serially.

An automated multi-unit composting facility for biodegradability evaluations

A system has been developed for studying the biodegradation of natural and synthetic polymeric material. The system is based on standard methods developed by the European Committee for Standardisation (CEN TC 261) (ISO/DIS 14855) and the American Society of Testing Materials, 'ASTM. Standard Test Method for Determining Aerobic. Biodegradation of Plastic Materials under Controlled Composting Conditions' (ASTM D 5338-92). A new low-cost compost facility has been used which satisfies the requirements of these standards. The system has been automated for data collection and has been run under the conditions specified by the standards. In the system, cellulose, newspaper and two starch-based polymers were treated with compost in a series of 3dm(3) vessels at 52 degreesC and under conditions of optimum moisture and pH. The degradation was followed over time by measuring the amount of carbon released as carbon dioxide. (C) 2001 Society of Chemical Industry.

The variety of variables in automated real-time refinement

The refinement calculus is a well-established theory for deriving program code from specifications. Recent research has extended the theory to handle timing requirements, as well as functional ones, and we have developed an interactive programming tool based on these extensions. Through a number of case studies completed using the tool, this paper explains how the tool helps the programmer by supporting the many forms of variables needed in the theory. These include simple state variables as in the untimed calculus, trace variables that model the evolution of properties over time, auxiliary variables that exist only to support formal reasoning, subroutine parameters, and variables shared between parallel processes.

Tool support for testing concurrent Java components

Concurrent programs are hard to test due to the inherent nondeterminism. This paper presents a method and tool support for testing concurrent Java components. Too[ support is offered through ConAn (Concurrency Analyser), a too] for generating drivers for unit testing Java classes that are used in a multithreaded context. To obtain adequate controllability over the interactions between Java threads, the generated driver contains threads that are synchronized by a clock. The driver automatically executes the calls in the test sequence in the prescribed order and compares the outputs against the expected outputs specified in the test sequence. The method and tool are illustrated in detail on an asymmetric producer-consumer monitor. Their application to testing over 20 concurrent components, a number of which are sourced from industry and were found to contain faults, is presented and discussed.

A framework and tool support for the systematic testing of model-based specifications

Formal specifications can precisely and unambiguously define the required behavior of a software system or component. However, formal specifications are complex artifacts that need to be verified to ensure that they are consistent, complete, and validated against the requirements. Specification testing or animation tools exist to assist with this by allowing the specifier to interpret or execute the specification. However, currently little is known about how to do this effectively. This article presents a framework and tool support for the systematic testing of formal, model-based specifications. Several important generic properties that should be satisfied by model-based specifications are first identified. Following the idea of mutation analysis, we then use variants or mutants of the specification to check that these properties are satisfied. The framework also allows the specifier to test application-specific properties. All properties are tested for a range of states that are defined by the tester in the form of a testgraph, which is a directed graph that partially models the states and transitions of the specification being tested. Tool support is provided for the generation of the mutants, for automatically traversing the testgraph and executing the test cases, and for reporting any errors. The framework is demonstrated on a small specification and its application to three larger specifications is discussed. Experience indicates that the framework can be used effectively to test small to medium-sized specifications and that it can reveal a significant number of problems in these specifications.

Tool support for executable documentation of Java class hierarchies

While object-oriented programming offers great solutions for today's software developers, this success has created difficult problems in class documentation and testing. In Java, two tools provide assistance: Javadoc allows class interface documentation to be embedded as code comments and JUnit supports unit testing by providing assert constructs and a test framework. This paper describes JUnitDoc, an integration of Javadoc and JUnit, which provides better support for class documentation and testing. With JUnitDoc, test cases are embedded in Javadoc comments and used as both examples for documentation and test cases for quality assurance. JUnitDoc extracts the test cases for use in HTML files serving as class documentation and in JUnit drivers for class testing. To address the difficult problem of testing inheritance hierarchies, JUnitDoc provides a novel solution in the form of a parallel test hierarchy. A small controlled experiment compares the readability of JUnitDoc documentation to formal documentation written in Object-Z. Copyright (c) 2005 John Wiley & Sons, Ltd.