Extending Automated Testing To High-level Software Requirements : A study on the feasibility of automated acceptance-testing

Automated acceptance testing is the testing of software done in higher level to test whether the system abides by the requirements desired by the business clients by the use of piece of script other than the software itself. This project is a study of the feasibility of acceptance tests written in Behavior Driven Development principle. The project includes an implementation part where automated accep- tance testing is written for Touch-point web application developed by Dewire (a software consultant company) for Telia (a telecom company) from the require- ments received from the customer (Telia). The automated acceptance testing is in Cucumber-Selenium framework which enforces Behavior Driven Development principles. The purpose of the implementation is to verify the practicability of this style of acceptance testing. From the completion of implementation, it was concluded that all the requirements from customer in real world can be converted into executable specifications and the process was not at all time-consuming or difficult for a low-experienced programmer like the author itself. The project also includes survey to measure the learnability and understandability of Gherkin- the language that Cucumber understands. The survey consist of some Gherkin exam- ples followed with questions that include making changes to the Gherkin exam- ples. Survey had 3 parts: first being easy, second medium and third most difficult. Survey also had a linear scale from 1 to 5 to rate the difficulty level for each part of the survey. 1 stood for very easy and 5 for very difficult. Time when the partic- ipants began the survey was also taken in order to calculate the total time taken by the participants to learn and answer the questions. Survey was taken by 18 of the employers of Dewire who had primary working role as one of the programmer, tester and project manager. In the result, tester and project manager were grouped as non-programmer. The survey concluded that it is very easy and quick to learn Gherkin. While the participants rated Gherkin as very easy.

A self -testing approach for autonomic software

As researchers and practitioners move towards a vision of software systems that configure, optimize, protect, and heal themselves, they must also consider the implications of such self-management activities on software reliability. Autonomic computing (AC) describes a new generation of software systems that are characterized by dynamically adaptive self-management features. During dynamic adaptation, autonomic systems modify their own structure and/or behavior in response to environmental changes. Adaptation can result in new system configurations and capabilities, which need to be validated at runtime to prevent costly system failures. However, although the pioneers of AC recognize that validating autonomic systems is critical to the success of the paradigm, the architectural blueprint for AC does not provide a workflow or supporting design models for runtime testing. ^ This dissertation presents a novel approach for seamlessly integrating runtime testing into autonomic software. The approach introduces an implicit self-test feature into autonomic software by tailoring the existing self-management infrastructure to runtime testing. Autonomic self-testing facilitates activities such as test execution, code coverage analysis, timed test performance, and post-test evaluation. In addition, the approach is supported by automated testing tools, and a detailed design methodology. A case study that incorporates self-testing into three autonomic applications is also presented. The findings of the study reveal that autonomic self-testing provides a flexible approach for building safe, reliable autonomic software, while limiting the development and performance overhead through software reuse. ^

BabelFuse data fusion unit with precision wireless clock synchronisation

A significant issue encountered when fusing data received from multiple sensors is the accuracy of the timestamp associated with each piece of data. This is particularly important in applications such as Simultaneous Localisation and Mapping (SLAM) where vehicle velocity forms an important part of the mapping algorithms; on fastmoving vehicles, even millisecond inconsistencies in data timestamping can produce errors which need to be compensated for. The timestamping problem is compounded in a robot swarm environment due to the use of non-deterministic readily-available hardware (such as 802.11-based wireless) and inaccurate clock synchronisation protocols (such as Network Time Protocol (NTP)). As a result, the synchronisation of the clocks between robots can be out by tens-to-hundreds of milliseconds making correlation of data difficult and preventing the possibility of the units performing synchronised actions such as triggering cameras or intricate swarm manoeuvres. In this thesis, a complete data fusion unit is designed, implemented and tested. The unit, named BabelFuse, is able to accept sensor data from a number of low-speed communication buses (such as RS232, RS485 and CAN Bus) and also timestamp events that occur on General Purpose Input/Output (GPIO) pins referencing a submillisecondaccurate wirelessly-distributed "global" clock signal. In addition to its timestamping capabilities, it can also be used to trigger an attached camera at a predefined start time and frame rate. This functionality enables the creation of a wirelessly-synchronised distributed image acquisition system over a large geographic area; a real world application for this functionality is the creation of a platform to facilitate wirelessly-distributed 3D stereoscopic vision. A ‘best-practice’ design methodology is adopted within the project to ensure the final system operates according to its requirements. Initially, requirements are generated from which a high-level architecture is distilled. This architecture is then converted into a hardware specification and low-level design, which is then manufactured. The manufactured hardware is then verified to ensure it operates as designed and firmware and Linux Operating System (OS) drivers are written to provide the features and connectivity required of the system. Finally, integration testing is performed to ensure the unit functions as per its requirements. The BabelFuse System comprises of a single Grand Master unit which is responsible for maintaining the absolute value of the "global" clock. Slave nodes then determine their local clock o.set from that of the Grand Master via synchronisation events which occur multiple times per-second. The mechanism used for synchronising the clocks between the boards wirelessly makes use of specific hardware and a firmware protocol based on elements of the IEEE-1588 Precision Time Protocol (PTP). With the key requirement of the system being submillisecond-accurate clock synchronisation (as a basis for timestamping and camera triggering), automated testing is carried out to monitor the o.sets between each Slave and the Grand Master over time. A common strobe pulse is also sent to each unit for timestamping; the correlation between the timestamps of the di.erent units is used to validate the clock o.set results. Analysis of the automated test results show that the BabelFuse units are almost threemagnitudes more accurate than their requirement; clocks of the Slave and Grand Master units do not di.er by more than three microseconds over a running time of six hours and the mean clock o.set of Slaves to the Grand Master is less-than one microsecond. The common strobe pulse used to verify the clock o.set data yields a positive result with a maximum variation between units of less-than two microseconds and a mean value of less-than one microsecond. The camera triggering functionality is verified by connecting the trigger pulse output of each board to a four-channel digital oscilloscope and setting each unit to output a 100Hz periodic pulse with a common start time. The resulting waveform shows a maximum variation between the rising-edges of the pulses of approximately 39¥ìs, well below its target of 1ms.

Predicting fault-prone software modules with rank sum classification

The detection and correction of defects remains among the most time consuming and expensive aspects of software development. Extensive automated testing and code inspections may mitigate their effect, but some code fragments are necessarily more likely to be faulty than others, and automated identification of fault prone modules helps to focus testing and inspections, thus limiting wasted effort and potentially improving detection rates. However, software metrics data is often extremely noisy, with enormous imbalances in the size of the positive and negative classes. In this work, we present a new approach to predictive modelling of fault proneness in software modules, introducing a new feature representation to overcome some of these issues. This rank sum representation offers improved or at worst comparable performance to earlier approaches for standard data sets, and readily allows the user to choose an appropriate trade-off between precision and recall to optimise inspection effort to suit different testing environments. The method is evaluated using the NASA Metrics Data Program (MDP) data sets, and performance is compared with existing studies based on the Support Vector Machine (SVM) and Naïve Bayes (NB) Classifiers, and with our own comprehensive evaluation of these methods.

Simple software processes and tests improve the reliability and usefulness of a model.

Models are abstractions of reality that have predetermined limits (often not consciously thought through) on what problem domains the models can be used to explore. These limits are determined by the range of observed data used to construct and validate the model. However, it is important to remember that operating the model beyond these limits, one of the reasons for building the model in the first place, potentially brings unwanted behaviour and thus reduces the usefulness of the model. Our experience with the Agricultural Production Systems Simulator (APSIM), a farming systems model, has led us to adapt techniques from the disciplines of modelling and software development to create a model development process. This process is simple, easy to follow, and brings a much higher level of stability to the development effort, which then delivers a much more useful model. A major part of the process relies on having a range of detailed model tests (unit, simulation, sensibility, validation) that exercise a model at various levels (sub-model, model and simulation). To underline the usefulness of testing, we examine several case studies where simulated output can be compared with simple relationships. For example, output is compared with crop water use efficiency relationships gleaned from the literature to check that the model reproduces the expected function. Similarly, another case study attempts to reproduce generalised hydrological relationships found in the literature. This paper then describes a simple model development process (using version control, automated testing and differencing tools), that will enhance the reliability and usefulness of a model.

Teknik för en flerskiktadwebbapplikation

The report analyses if some common problems can be avoided by using modern technology. As a reference system “Fartygsrapporteringssystemet” is used. It is an n-tier web application built with modern technology at time, 2003-2004. The aim is to examine whether ASP.Net MVC, Windows Communication Foundation, Workflow Foundation and SQL Server 2005 Service Broker can be used to create an n-tier web application which also communicate with other systems and facilitate automated testing. The report describes the construction of a prototype in which the presentation layer uses ASP.Net MVC to separate presentation and business logic. Communication with the business layer is done through the Windows Communication Foundation. Hard coded processes are broken out and dealt with by Workflow Foundation. Asynchronous communication with other systems is done by using Microsoft SQL Server 2005 Service Broker. The results of the analysis is that these techniques can be used to create a n-tier web application, but that ASP.Net MVC, which at present only available in a preview release, is not sufficiently developed yet.

Testverktyg för test av mobila applikationer : En jämförande fallstudie

Användandet av mobila applikationer har växt radikalt de senaste åren och de samverkar med många system. Därför ställs det högre krav på kvaliteten och att applikationen ska anpassas till många olika enheter, operativsystem samt plattformar. Detta gör att test av mobila applikationer blivit viktigare och större. Detta arbete har bedrivits som en jämförande fallstudie inom området test av mobila applikationer samt testverktyg. Syftet har varit att beskriva hur testning av mobila applikationer sker idag vilket gjorts genom litteraturstudier och intervjuer med IT-företag. Ett annat syfte har varit att utvärdera fyra testverktyg, deras för- och nackdelar samt hur de kan användas vid testning av mobila applikationer och jämföras mot manuell testning utan testverktyg. Detta har gjorts genom att skapa förstahandserfarenheter baserat på användandet av testverktygen. Under arbetet har vi utgått från mobila applikationer som vi fått tillgång till av Triona, som varit vår samarbetspartner.Idag finns många olika testverktyg som kan användas som stöd för testningen men få företag har implementerat något eftersom det kräver både tid och kompetens samt valet av testverktyg kan vara svårt. Testverktygen har olika för- och nackdelar vilket gör att de passar olika bra beroende på typ av projekt och applikation. Fördelar med att använda testverktyg är möjligheten att kunna automatisera, testa på flera enheter samtidigt samt få tillgång till enheter via molnet. Utmaningarna är att det kan vara svårt att installera och lära sig testverktyget samt att licenserna kan vara dyra. Det är därför viktigt att redan innan implementationen veta vilka tester och applikationer testverktygen ska användas till samt vem som ska använda det. Utifrån vår studie kan slutsatsen dras att inget testverktyg är helt komplett men de kan bidra med olika funktioner vilket effektiviserar delar av testningen av mobila applikationer.

A preliminary survey of factors affecting software testers

Most software testing research has focused on the development of systematic, standardised, and automated testing methodologies and tools. The abilities and expertise needed to apply such techniques and tools - such as personality traits, education, and experience - have attracted a comparatively small amount of research attention. However, the limited research in the area to date provides some indication that the human traits of software testers are important for effective testing. This paper presents the opinions of software testers themselves, collected through an online survey, on the importance of a variety of factors that influence effective testing, including testing-specific training, experience, skills, and human qualities like dedication and general intelligence. The survey responses strongly suggest that while testing tools and training are important, human factors were similarly considered highly important. Domain knowledge, experience, intelligence, and dedication, amongst other traits, were considered crucial for a software tester to be effective. As such, while systematic methodologies are important, the individual most clearly does matter in software testing. The results of our research have implications for education, recruitment, training and management of software testers.

Soluções globais para sistemas industriais-teste de software

Relatório de estágio apresentada para cumprimento dos requisitos necessários à obtenção do grau de Mestre em Sistemas de Informação Organizacionais

Tool support for executable documentation of Java class hierarchies

While object-oriented programming offers great solutions for today's software developers, this success has created difficult problems in class documentation and testing. In Java, two tools provide assistance: Javadoc allows class interface documentation to be embedded as code comments and JUnit supports unit testing by providing assert constructs and a test framework. This paper describes JUnitDoc, an integration of Javadoc and JUnit, which provides better support for class documentation and testing. With JUnitDoc, test cases are embedded in Javadoc comments and used as both examples for documentation and test cases for quality assurance. JUnitDoc extracts the test cases for use in HTML files serving as class documentation and in JUnit drivers for class testing. To address the difficult problem of testing inheritance hierarchies, JUnitDoc provides a novel solution in the form of a parallel test hierarchy. A small controlled experiment compares the readability of JUnitDoc documentation to formal documentation written in Object-Z. Copyright (c) 2005 John Wiley & Sons, Ltd.

Optimization and improvement of NSCU 1.0. Universal Functional Tester produced by Evoleo Technologi

The aim of the project was to improve an existing testing machine that is produced by the company EVOLEO Technologies. New conceptions of each part have been invented in order to produce an innovative unit that combines optimal segments from the old construction with the new, improved ones. The machine is meant to be testing different kind of devices that use specific elements like: buttons, knobs, monitors. The main purpose is to create various concepts of components that could be changed in order to lower the cost, weight or to simplify the operating process. Figure 1. shows the already existing discussed device.

Virtual fencing applications : Implementing and testing an automated cattle control system

Managing livestock movement in extensive systems has environmental and production benefits. Currently permanent wire fencing is used to control cattle; this is both expensive and inflexible. Cattle are known to respond to auditory and visual cues and we investigated whether these can be used to manipulate their behaviour. Twenty-five Belmont Red steers with a mean live weight of 270kg were each randomly assigned to one of five treatments. Treatments consisted of a combination of cues (audio, tactile and visual stimuli) and consequence (electrical stimulation). The treatments were electrical stimulation alone, audio plus electrical stimulation, vibration plus electrical stimulation, light plus electrical stimulation and electrified electric fence (6kV) plus electrical stimulation. Cue stimuli were administered for 3s followed immediately by electrical stimulation (consequence) of 1kV for 1s. The experiment tested the operational efficacy of an on-animal control or virtual fencing system. A collar-halter device was designed to carry the electronics, batteries and equipment providing the stimuli, including audio, vibration, light and electrical of a prototype virtual fencing device. Cattle were allowed to travel along a 40m alley to a group of peers and feed while their rate of travel and response to the stimuli were recorded. The prototype virtual fencing system was successful in modifying the behaviour of the cattle. The rate of travel of cattle along the alley demonstrated the large variability in behavioural response associated with tactile, visual and audible cues. The experiment demonstrated virtual fencing has potential for controlling cattle in extensive grazing systems. However, larger numbers of cattle need to be tested to derive a better understanding of the behavioural variance. Further controlled experimental work is also necessary to quantify the interaction between cues, consequences and cattle learning.

automated test case generation for the stress testing of multimedia systems

With the advancement in network bandwidth and computing power, multimedia systems have become a popular means for information delivery. However, general principles of system testing cannot be directly applied to testing of multimedia systems on account of their stringent temporal and synchronization requirements. In particular, few studies have been made on the stress testing of multimedia systems with respect to their temporal requirements under resource saturation. Stress testing is important because erroneous behavior is most likely to occur under resource saturation. This paper presents an automatable method of test case generation for the stress testing of multimedia systems. It adapts constraint solving techniques to generate test cases that lead to potential resource saturation in a multimedia system. Coverage of the test cases is defined upon the reachability graph of a multimedia system. The proposed stress testing technique is supported by tools and has been successfully applied to a real-life commercial multimedia system. Although our technique focuses on the stress testing of multimedia systems, the underlying issues and concepts are applicable to other types of real-time systems.

Validation of an automated algorithm for auditory brainstem response testing

This paper discusses the auditory brainstem response (ABR) testing for infants.