User dyads in software testing: bypassing the need for expert observers

This paper arises out of a research study into the online help facilities provided in popular software applications such as word processors. Its particular focus is on experimental methods of evaluating the effectiveness and usability of those facilities. Focus groups, questionnaires, and online surveys had already been used in other phases of the study, but it was judged that these approaches would be unsuitable for measuring effectiveness and usability because they are susceptible to respondents' subjectivity. Direct observation of people working on set word-processing tasks was ruled out initially because of a lack of trained observers; it would have taken too long for the investigator to observe a large enough sample by himself. Automatic recording of users' actions was also rejected, as it would have demanded equipment and/or software that was not available and seemed too expensive to acquire. The approach and techniques described here were an attempt to overcome these difficulties by using observers drawn from the same population of students that provided the test subjects; as a by-product, this may also have enhanced the acceptability (and hence possibly the validity) of the experiments by reducing the exam pressure perceived by participants.

Supporting the software testing process through specification animation

Achieving consistency between a specification and its implementation is an important part of software development. In this paper, we present a method for generating passive test oracles that act as self-checking implementations. The implementation is verified using an animation tool to check that the behavior of the implementation matches the behavior of the specification. We discuss how to integrate this method into a framework developed for systematically animating specifications, which means a tester can significantly reduce testing time and effort by reusing work products from the animation. One such work product is a testgraph: a directed graph that partially models the states and transitions of the specification. Testgraphs are used to generate sequences for animation, and during testing, to execute these same sequences on the implementation.

Molecular clocks in reptiles: Life history influences rate of molecular evolution

Life history has been implicated as a determinant of variation in rate of molecular evolution amongst vertebrate species because of a negative correlation between bode size and substitution rate for many Molecular data sets. Both the generality and the cause of the negative bode size trend have been debated, and the validity of key studies has been questioned (particularly concerning the failure to account for phylogenetic bias). In this study, a comparative method has been used to test for an association between a range of life-history variables-such as body size age at maturity, and clutch size-and DNA substitution rate for three genes (NADH4, cytochrome b, and c-mos). A negative relationship between body size and rate of molecular evolution was found for phylogenetically independent pairs of reptile species spanning turtles. lizards. snakes, crocodile, and tuatara. Although this Study was limited by the number of comparisons for which both sequence and lite-history data were available, the results, suggest that a negative bode size trend in rate of molecular evloution may be a general feature of reptile molecular evolution. consistent with similar studies of mammals and birds. This observation has important implications for uncovering the mechanisms of molecular evolution and warns against assuming that related lineages will share the same substitution rate (a local molecular clock) in order to date evolutionary divergences from DNA sequences.

Rates of gene rearrangement and nucleotide substitution are correlated in the mitochondrial genomes of insects

A number of studies indicated that lineages of animals with high rates of mitochondrial (mt) gene rearrangement might have high rates of mt nucleotide substitution. We chose the hemipteroid assemblage and the Insecta to test the idea that rates of mt gene rearrangement and mt nucleotide substitution are correlated. For this purpose, we sequenced the mt genome of a lepidopsocid from the Psocoptera, the only order of hemipteroid insects for which an entire mtDNA sequence is not available. The mt genome of this lepidopsocid is circular, 16,924 bp long, and contains 37 genes and a putative control region; seven tRNA genes and a protein-coding gene in this genome have changed positions relative to the ancestral arrangement of mt genes of insects. We then compared the relative rates of nucleotide substitution among species from each of the four orders of hemipteroid insects and among the 20 insects whose mt genomes have been sequenced entirely. All comparisons among the hernipteroid insects showed that species with higher rates of gene rearrangement also had significantly higher rates of nucleotide substitution statistically than did species with lower rates of gene rearrangement. In comparisons among the 20 insects, where the mt genomes of the two species differed by more than five breakpoints, the more rearranged species always had a significantly higher rate of nucleotide substitution than the less rearranged species. However, in comparisons where the mt genomes of two species differed by five or less breakpoints, the more rearranged species did not always have a significantly higher rate of nucleotide substitution than the less rearranged species. We tested the statistical significance of the correlation between the rates of mt gene rearrangement and mt nucleotide substitution with nine pairs of insects that were phylogenetically independent from one 2 another. We found that the correlation was positive and statistically significant (R-2 = 0.73, P = 0.01; R-s = 0.67, P < 0.05). We propose that increased rates of nucleotide substitution may lead to increased rates of gene rearrangement in the mt genomes of insects.

Sociality and the rate of molecular evolution

The molecular clock does not tick at a uniform rate in all taxa but maybe influenced by species characteristics. Eusocial species (those with reproductive division of labor) have been predicted to have faster rates of molecular evolution than their nonsocial relatives because of greatly reduced effective population size; if most individuals in a population are nonreproductive and only one or few queens produce all the offspring, then eusocial animals could have much lower effective population sizes than their solitary relatives, which should increase the rate of substitution of nearly neutral mutations. An earlier study reported faster rates in eusocial honeybees and vespid wasps but failed to correct for phylogenetic nonindependence or to distinguish between potential causes of rate variation. Because sociality has evolved independently in many different lineages, it is possible to conduct a more wide-ranging study to test the generality of the relationship. We have conducted a comparative analysis of 25 phylogenetically independent pairs of social lineages and their nonsocial relatives, including bees, wasps, ants, termites, shrimps, and mole rats, using a range of available DNA sequences (mitochondrial and nuclear DNA coding for proteins and RNAs, and nontranslated sequences). By including a wide range of social taxa, we were able to test whether there is a general influence of sociality on rates of molecular evolution and to test specific predictions of the hypothesis: (1) that social species have faster rates because they have reduced effective population sizes; (2) that mitochondrial genes would show a greater effect of sociality than nuclear genes; and (3) that rates of molecular evolution should be correlated with the degree of sociality. We find no consistent pattern in rates of molecular evolution between social and nonsocial lineages and no evidence that mitochondrial genes show faster rates in social taxa. However, we show that the most highly eusocial Hymenoptera do have faster rates than their nonsocial relatives. We also find that social parasites (that utilize the workers from related species to produce their own offspring) have faster rates than their social relatives, which is consistent with an effect of lower effective population size on rate of molecular evolution. Our results illustrate the importance of allowing for phylogenetic nonindependence when conducting investigations of determinants of variation in rate of molecular evolution.

UML approach to the generation of test sequences for Java-based concurrent systems

Starting with a UML specification that captures the underlying functionality of some given Java-based concurrent system, we describe a systematic way to construct, from this specification, test sequences for validating an implementation of the system. The approach is to first extend the specification to create UML state machines that directly address those aspects of the system we wish to test. To be specific, the extended UML state machines can capture state information about the number of waiting threads or the number of threads blocked on a given object. Using the SAL model checker we can generate from the extended UML state machines sequences that cover all the various possibilities of events and states. These sequences can then be directly transformed into test sequences suitable for input into a testing tool such as ConAn. As an illustration, the methodology is applied to generate sequences for testing a Java implementation of the producer-consumer system. © 2005 IEEE

A framework for table driven testing of Java classes

With the advent of object-oriented languages and the portability of Java, the development and use of class libraries has become widespread. Effective class reuse depends on class reliability which in turn depends on thorough testing. This paper describes a class testing approach based on modeling each test case with a tuple and then generating large numbers of tuples to thoroughly cover an input space with many interesting combinations of values. The testing approach is supported by the Roast framework for the testing of Java classes. Roast provides automated tuple generation based on boundary values, unit operations that support driver standardization, and test case templates used for code generation. Roast produces thorough, compact test drivers with low development and maintenance cost. The framework and tool support are illustrated on a number of non-trivial classes, including a graphical user interface policy manager. Quantitative results are presented to substantiate the practicality and effectiveness of the approach. Copyright (C) 2002 John Wiley Sons, Ltd.

Aberrant protective force generation during neural provocation testing and the effect of treatment in patients with neurogenic cervicobrachial pain

Background: Observation of the occurrence of protective muscle activity is advocated in assessment of the peripheral nervous system by means of neural provocation tests. However, no studies have yet demonstrated abnormal force generation in a patient population. Objectives: To analyze whether aberrations in shoulder girdle-elevation force during neural tissue provocation testing for the median nerve (NTPTI) can be demonstrated, and whether possible aberrations can be normalized following cervical mobilization. Study Design: A single-blind randomized comparative controlled study. Setting: Laboratory setting annex in a manual therapy teaching practice. Participants: Twenty patients with unilateral or bilateral neurogenic cervicobrachial pain. Methods: During the NTPTI, we used a load cell and electrogoniometer to record continuously the shoulder-girdle elevation force in relation to the available range of elbow extension. Following randomization, we analyzed the immediate treatment effects of a cervical contralateral lateral glide mobilization technique (experimental group) and therapeutic ultrasound (control group). Results: On the involved side, the shoulder-girdle elevation force occur-red earlier, and the amount of force at the end of the test was substantially, though not significantly, greater than that on the uninvolved side at the corresponding range of motion. Together with a significant reduction in pain perception after cervical mobilization, a clear tendency toward normalization of the force curve could be observed, namely, a significant decrease in force generation and a delayed onset. The control group demonstrated no differences. Conclusions: Aberrations in force generation during neural, provocation testing are present in patients with neurogenic pain and can be normalized with appropriate treatment modalities.

A framework and tool support for the systematic testing of model-based specifications

Formal specifications can precisely and unambiguously define the required behavior of a software system or component. However, formal specifications are complex artifacts that need to be verified to ensure that they are consistent, complete, and validated against the requirements. Specification testing or animation tools exist to assist with this by allowing the specifier to interpret or execute the specification. However, currently little is known about how to do this effectively. This article presents a framework and tool support for the systematic testing of formal, model-based specifications. Several important generic properties that should be satisfied by model-based specifications are first identified. Following the idea of mutation analysis, we then use variants or mutants of the specification to check that these properties are satisfied. The framework also allows the specifier to test application-specific properties. All properties are tested for a range of states that are defined by the tester in the form of a testgraph, which is a directed graph that partially models the states and transitions of the specification being tested. Tool support is provided for the generation of the mutants, for automatically traversing the testgraph and executing the test cases, and for reporting any errors. The framework is demonstrated on a small specification and its application to three larger specifications is discussed. Experience indicates that the framework can be used effectively to test small to medium-sized specifications and that it can reveal a significant number of problems in these specifications.

Formal derivation of finite state machines for class testing

Previous work on generating state machines for the purpose of class testing has not been formally based. There has also been work on deriving state machines from formal specifications for testing non-object-oriented software. We build on this work by presenting a method for deriving a state machine for testing purposes from a formal specification of the class under test. We also show how the resulting state machine can be used as the basis for a test suite developed and executed using an existing framework for class testing. To derive the state machine, we identify the states and possible interactions of the operations of the class under test. The Test Template Framework is used to formally derive the states from the Object-Z specification of the class under test. The transitions of the finite state machine are calculated from the derived states and the class's operations. The formally derived finite state machine is transformed to a ClassBench testgraph, which is used as input to the ClassBench framework to test a C++ implementation of the class. The method is illustrated using a simple bounded queue example.

A framework for specification-based testing

Test templates and a test template framework are introduced as useful concepts in specification-based testing. The framework can be defined using any model-based specification notation and used to derive tests from model-based specifications-in this paper, it is demonstrated using the Z notation. The framework formally defines test data sets and their relation to the operations in a specification and to other test data sets, providing structure to the testing process. Flexibility is preserved, so that many testing strategies can be used. Important application areas of the framework are discussed, including refinement of test data, regression testing, and test oracles.

Testing the link between the latitudinal gradient in species richness and rates of molecular evolution

Numerous hypotheses have been proposed to explain latitudinal gradients in species richness, but all are subject to ongoing debate. Here we examine Rohde's (1978, 1992) hypothesis, which proposes that climatic conditions at low latitudes lead to elevated rates of speciation. This hypothesis predicts that rates of molecular evolution should increase towards lower latitudes, but this prediction has never been tested. We discuss potential links between rates of molecular evolution and latitudinal diversity gradients, and present the first test of latitudinal variation in rates of molecular evolution. Using 45 phylogenetically independent, latitudinally separated pairs of bird species and higher taxa, we compare rates of evolution of two mitochondrial genes and DNA-DNA hybridization distances. We find no support for an effect of latitude on rate of molecular evolution. This result casts doubt on the generality of a key component of Rohde's hypothesis linking climate and speciation.

Tool support for testing concurrent Java components

Concurrent programs are hard to test due to the inherent nondeterminism. This paper presents a method and tool support for testing concurrent Java components. Too[ support is offered through ConAn (Concurrency Analyser), a too] for generating drivers for unit testing Java classes that are used in a multithreaded context. To obtain adequate controllability over the interactions between Java threads, the generated driver contains threads that are synchronized by a clock. The driver automatically executes the calls in the test sequence in the prescribed order and compares the outputs against the expected outputs specified in the test sequence. The method and tool are illustrated in detail on an asymmetric producer-consumer monitor. Their application to testing over 20 concurrent components, a number of which are sourced from industry and were found to contain faults, is presented and discussed.

Viewpoint-based testing of concurrent components

The use of multiple partial viewpoints is recommended for specification. We believe they also can be useful for devising strategies for testing. In this paper, we use Object-Z to formally specify concurrent Java components from viewpoints based on the separation of application and synchronisation concerns inherent in Java monitors. We then use the Test-Template Framework on the Object-Z viewpoints to devise a strategy for testing the components. When combining the test templates for the different viewpoints we focus on the observable behaviour of the application to systematically derive a practical testing strategy. The Producer-Consumer and Readers-Writers problems are considered as case studies.

Integrating formal specification and software verification and validation

It is not surprising that students are unconvinced about the benefits of formal methods if we do not show them how these methods can be integrated with other activities in the software lifecycle. In this paper, we describe an approach to integrating formal specification with more traditional verification and validation techniques in a course that teaches formal specification and specification-based testing. This is accomplished through a series of assignments on a single software component that involves specifying the component in Object-Z, validating that specification using inspection and a specification animation tool, and then testing an implementation of the specification using test cases derived from the formal specification.