A case study in specification and implementation testing

Achieving consistency between a specification and its implementation is an important part of software development In previous work, we have presented a method and tool support for testing a formal specification using animation and then verifying an implementation of that specification. The method is based on a testgraph, which provides a partial model of the application under test. The testgraph is used in combination with an animator to generate test sequences for testing the formal specification. The same testgraph is used during testing to execute those same sequences on the implementation and to ensure that the implementation conforms to the specification. So far, the method and its tool support have been applied to software components that can be accessed through an application programmer interface (API). In this paper, we use an industrially-based case study to discuss the problems associated with applying the method to a software system with a graphical user interface (GUI). In particular, the lack of a standardised interface, as well as controllability and observability problems, make it difficult to automate the testing of the implementation. The method can still be applied, but the amount of testing that can be carried on the implementation is limited by the manual effort involved.

Testing java interrupts and timed waits

Testing concurrent software is difficult due to problems with inherent nondeterminism. In previous work, we have presented a method and tool support for the testing of concurrent Java components. In this paper, we extend that work by presenting and discussing techniques for testing Java thread interrupts and timed waits. Testing thread interrupts is important because every Java component that calls wait must have code dealing with these interrupts. For a component that uses interrupts and timed waits to provide its basic functionality, the ability to test these features is clearly even more important. We discuss the application of the techniques and tool support to one such component, which is a nontrivial implementation of the readers-writers problem.

Dealing with non-determinism in testing concurrent java components

The testing of concurrent software components can be difficult due to the inherent non-determinism present in these components. For example, if the same test case is run multiple times, it may produce different results. This non-determinism may lead to problems with determining expected outputs. In this paper, we present and discuss several possible solutions to this problem in the context of testing concurrent Java components using the ConAn testing tool. We then present a recent extension to the tool that provides a general solution to this problem that is sufficient to deal with the level of non-determinism that we have encountered in testing over 20 components with ConAn. © 2005 IEEE

Test strategies on functionally partitioned module-based programmable architecture for base-band processing

A specialised reconfigurable architecture for telecommunication base-band processing is augmented with testing resources. The routing network is linked via virtual wire hardware modules to reduce the area occupied by connecting buses. The number of switches within the routing matrices is also minimised, which increases throughput without sacrificing flexibility. The testing algorithm was developed to systematically search for faults in the processing modules and the flexible high-speed routing network within the architecture. The testing algorithm starts by scanning the externally addressable memory space and testing the master controller. The controller then tests every switch in the route-through switch matrix by making loops from the shared memory to each of the switches. The local switch matrix is also tested in the same way. Next the local memory is scanned. Finally, pre-defined test vectors are loaded into local memory to check the processing modules. This algorithm scans all possible paths within the interconnection network exhaustively and reports all faults. Strategies can be inserted to bypass minor faults

Tool support for generating passive C++ test oracles from object-z specifications

A test oracle provides a means for determining whether an implementation behaves according to its specification. A passive test oracle checks that the correct behaviour has been implemented, but does not implement the behaviour itself. In previous work, we have presented a method that allows us to derive passive C++ test oracles from formal specifications written in Object-Z. We describe the "Warlock" prototype tool that supports the method. Warlock is built on top of an existing Object-Z type checker and generates oracle code for a substantial subset of the Object-Z language. We describe the architecture of Warlock and its application to a number of Object-Z specifications. We also discuss its current limitations.

Improving wheat simulation capabilities in Australia from a cropping systems perspective - II. Testing simulation capabilities of wheat growth

To simulate cropping systems, crop models must not only give reliable predictions of yield across a wide range of environmental conditions, they must also quantify water and nutrient use well, so that the status of the soil at maturity is a good representation of the starting conditions for the next cropping sequence. To assess the suitability for this task a range of crop models, currently used in Australia, were tested. The models differed in their design objectives, complexity and structure and were (i) tested on diverse, independent data sets from a wide range of environments and (ii) model components were further evaluated with one detailed data set from a semi-arid environment. All models were coded into the cropping systems shell APSIM, which provides a common soil water and nitrogen balance. Crop development was input, thus differences between simulations were caused entirely by difference in simulating crop growth. Under nitrogen non-limiting conditions between 73 and 85% of the observed kernel yield variation across environments was explained by the models. This ranged from 51 to 77% under varying nitrogen supply. Water and nitrogen effects on leaf area index were predicted poorly by all models resulting in erroneous predictions of dry matter accumulation and water use. When measured light interception was used as input, most models improved in their prediction of dry matter and yield. This test highlighted a range of compensating errors in all modelling approaches. Time course and final amount of water extraction was simulated well by two models, while others left up to 25% of potentially available soil water in the profile. Kernel nitrogen percentage was predicted poorly by all models due to its sensitivity to small dry matter changes. Yield and dry matter could be estimated adequately for a range of environmental conditions using the general concepts of radiation use efficiency and transpiration efficiency. However, leaf area and kernel nitrogen dynamics need to be improved to achieve better estimates of water and nitrogen use if such models are to be use to evaluate cropping systems. (C) 1998 Elsevier Science B.V.

Evolution of the genetic covariance between male and female components of mate recognition: An experimental test

The evolution of a positive genetic correlation between male and female components of mate recognition systems will result as a consequence of assortative mating and, in particular, is central to a number of theories of sexual selection. Although the existence of such genetic correlations has been investigated in a number of taxa, it has yet to be shown that such correlations evolve and whether they may evolve as rapidly as suggested by sexual selection models. In this study, I used a hybridization experiment to disrupt natural mate recognition systems and then observed the subsequent evolutionary dynamics of the genetic correlation between male and female components for 56 generations in hybrids between Drosophila serrata and Drosophila birchii. The genetic correlation between male and female components evolved from 0.388 at generation 5 to 1.017 at generation 37 and then declined to -0.040 after a further 19 generations. These results indicated that the genetic basis of the mate recognition system in the hybrid populations evolved rapidly. The initial rapid increase in the genetic correlation was consistent with the classic assumption that male and female components will coevolve under sexual selection. The subsequent decline in genetic correlation may be attributable to the fixation of major genes or, alternatively, may be a result of a cyclic evolutionary change in mate recognition.

A landscape-scale test of the predictive ability of a spatially explicit model for population viability analysis

1. Although population viability analysis (PVA) is widely employed, forecasts from PVA models are rarely tested. This study in a fragmented forest in southern Australia contrasted field data on patch occupancy and abundance for the arboreal marsupial greater glider Petauroides volans with predictions from a generic spatially explicit PVA model. This work represents one of the first landscape-scale tests of its type. 2. Initially we contrasted field data from a set of eucalypt forest patches totalling 437 ha with a naive null model in which forecasts of patch occupancy were made, assuming no fragmentation effects and based simply on remnant area and measured densities derived from nearby unfragmented forest. The naive null model predicted an average total of approximately 170 greater gliders, considerably greater than the true count (n = 81). 3. Congruence was examined between field data and predictions from PVA under several metapopulation modelling scenarios. The metapopulation models performed better than the naive null model. Logistic regression showed highly significant positive relationships between predicted and actual patch occupancy for the four scenarios (P = 0.001-0.006). When the model-derived probability of patch occupancy was high (0.50-0.75, 0.75-1.00), there was greater congruence between actual patch occupancy and the predicted probability of occupancy. 4. For many patches, probability distribution functions indicated that model predictions for animal abundance in a given patch were not outside those expected by chance. However, for some patches the model either substantially over-predicted or under-predicted actual abundance. Some important processes, such as inter-patch dispersal, that influence the distribution and abundance of the greater glider may not have been adequately modelled. 5. Additional landscape-scale tests of PVA models, on a wider range of species, are required to assess further predictions made using these tools. This will help determine those taxa for which predictions are and are not accurate and give insights for improving models for applied conservation management.

Impact of on-site testing for maternal syphilis on treatment delays, treatment rates, and perinatal mortality in rural South Africa: a randomised controlled trial

Background: Syphilis remains a significant cause of preventable perinatal death in developing countries with many women remaining untested and thus untreated. Syphilis testing in the clinic (on-site testing) may be a useful strategy to overcome this. We studied the impact of on-site syphilis testing on treatment delays and rates, and perinatal mortality. Methods: We conducted a cluster randomised controlled trial among seven pairs of primary healthcare clinics in rural South Africa, comparing on-site testing complemented by laboratory confirmation versus laboratory testing alone. Intervention clinics used the on-site test conducted by primary care nurses, with results and treatment available within an hour. Control clinics sent blood samples to the provincial laboratory, with results returned 2 weeks later. Results: Of 7134 women seeking antenatal care with available test results, 793 (11.1%) tested positive for syphilis. Women at intervention clinics completed treatment 16 days sooner on average (95% confidence interval: 11 to 21), though there was no significant difference in the proportion receiving adequate treatment at intervention (64%) and control (69%) clinics. There was also no significant difference in the proportion experiencing perinatal loss (3.3% v 5.1%; adjusted risk difference: -0.9%; 95% Cl -4.4 to 2.7). Conclusions: Despite reducing treatment delays, the addition of on-site syphilis testing to existing laboratory testing services did not lead to higher treatment rates or reduce perinatal mortality. However on-site testing for syphilis may remain an important option for improving antenatal care in settings where laboratory facilities are not available.

Whispered voice test for screening for hearing impairment in adults and children: systematic review

Objective To determine the accuracy of the whispered voice test in detecting hearing impairment in adults and children. Design Systematic review of studies of test accuracy. Data sources Medline, Embase, Science Citation Index, unpublished theses, manual searching of bibliographies of known primary and review articles, and contact with authors. Study selection Two reviewers independently selected and extracted data on study characteristics, quality, and accuracy of studies. Studies were included if they had cross sectional designs, at least one of the index tests was the whispered voice test, and the reference test (audiometry) was performed on at least 80% of the participants. Data extraction Data were used to form 2x2 contingency tables with hearing impairment by audiometry as the reference standard. Data synthesis The eight studies that were found used six different techniques. The sensitivity in the four adult studies was 90% or 100% and the specificity was 70% to 87%. The sensitivity in the four childhood studies ranged from 80% to 96% and specificity ranged from 90% to 98%. Conclusion The whispered voice test is a simple and accurate test for detecting hearing impairment. There is some concern regarding the lower sensitivity in children and the overall reproducibility of the test, particularly in primary care settings. Further studies should be conducted in primary care settings to explore the influence of components of the testing procedure to optimise test sensitivity and to promote standardisation of the testing procedure.