Integrating testcase generator into Eclipse environment

Tämädiplomityö tutkii kuinka Eclipse -ympäristöä voidaan käyttää testitapausten generoinnissa. Eräs diplomityön pääaiheista on tutkia voidaanko olemassa olevilla Eclipsen komponenteilla parantaa symboolitietoutta, jotta testitapausten generointiin saataisiin lisää tietoa. Aluksi diplomityö antaa lyhyen katsauksen ohjelmistojentestaukseen, jotta lukija ymmärtää mitä ohjelmistotekniikan osa-aluetta diplomityö käsittelee. Tämän jälkeen kerrotaan lisää tietoa itse testitapausten generointiprosessista. Kun perusteet on käsitelty, tutustetaan lukija Eclipse -ympäristöön, mikä se on, mistä se koostuu ja mitä sillä voidaan tehdä. Tarkempaa tietoa kerrotaan Eclipsen komponenteista joita voidaan käyttää apuna testitapausten generoinnissa. Integrointi esimerkkinä diplomityössä esitellään valmiin testitapausgeneraattorin integrointi Eclipse -ympäristöön. Lopuksi Eclipse -pohjaista ratkaisua verrataan symboolitietouden sekä ajoajan kannalta aikaisempaan ratkaisuun. Diplomityön tuloksena syntyi prototyyppi jonka avulla todistettiin, että Eclipse - ympäristöön on mahdollista integroida testitapausgeneraattori ja että se voi lisätä symboolitietoutta. Tämätietouden lisäys kuitenkin lisäsi myös tarvittavaa ajoaikaa, joissakintapauksissa jopa merkittävästi. Samalla todettiin, että tällä hetkellä on menossa projekteja joiden tarkoituksena on parantaa käytettyjen Eclipse komponenttien suorituskykyä ja että tämä voi parantaa tuloksia tulevaisuudessa.

Functional testing in TETRA system

Terrestrial Trunked Radio (TETRA) on moderni digitaalinen matkapuhelinjärjestelmän standardi, joka on suunniteltu täyttämään erityisesti viranomaisten vaativat tarpeet turvallisuuden ja luotettavuuden suhteen. Ohjelmiston testaus on tärkeä osa sen laadun takaamiseksi. Testaus on jaettu useisiin vaiheisiin ja se kattaa koko ohjelmiston elinkaaren: ohjelmiston kehittelystä alkaen asiakkaalle lähetettyyn valmiiseen tuotteeseen saakka. Toiminnallisuustestauksen suorittaa joko ohjelmiston suunnittelijat tai erillinen testausryhmä käyttäen Nokia TETRA-järjestelmän testauslaboratoriota. Testauksen tarkoituksena on varmistaa, että ohjelmisto, sen aliohjelmat ja ominaisuudet täyttävät niille annetut toiminnalliset ja laadulliset vaatimukset. Tämä diplomityö antaa yleiskuvan toiminnallisuustestausprosessista Nokia TETRA järjestelmän laboratoriossa. Se tarjoaa esimerkkitestitapauksen avulla kokonaiskuvan siitä, kuinka toiminnallisuustestausprosessi suoritetaan alusta loppuun.

New TTCN interfaces for WCDMA base station testing

WCDMA tukiasema (Node B) on osa UMTS-järjestelmän radioverkkoa. Node B on tärkeä verkkoelementti, jonka tarkoituksena on yhdistää mobiilikäyttäjät verkkoon. Telecom –ohjelmisto (TCOM SW) on vastuussa suuresta osasta Node B:n toiminnallisuutta. TCOM SW:n testaukseen käytetään paljon resursseja, jotta ohjelmiston oikeasta toiminnasta ja laadusta voidaan varmistua. System component testing on testausvaihe, jossa järjestelmän (Node B) osa (system component, tässä diplomityössä TCOM SW) testataan ennen sen integroimista muuhun järjestelmään. Tähän tarvitaan testityökalu ja testitapausten toteutus. Node B TTCN Tester (testeri) on työkalu, jota käytetään Node B:n ohjelmiston testauksessa. Testitapaukset toteutetaan TTCN-testinotaatiota käyttäen ja testataan testerin avulla. TCOM SW:n system component –testausvaihetta varten testeriin lisättiin uudet rajapinnat, joiden avulla voidaan simuloita Node B:n ATM-ohjelmistoa sekä WPA- ja WTR-yksiköitä. Tässä diplomityössä toteuttiin TTCN testitapaukset uusille rajapinnoille. Testitapaukset tekivät TCOM SW system component –testausvaiheen riippumattomaksi Node B:n ATM-ohjelmistosta sekä WPA- ja WTR-yksiköistä. Lisäksi TCOM SW:n toiminnan testaus näissä rajapinnoissa voidaan tästä lähtien tehdä automaattisesti. Testitapauksien toiminta varmistettiin testeriä käyttäen. Tulokset olivat hyviä, uudet testitapaukset ja TTCN rajapinnat toimivat oikein lisäten testauksen tehokkuutta.

Physical layer development in third generation protocol software architecture

Diplomityön tavoitteena oli kehittää kolmannen sukupolven fyysistä protokollakerrosta matkapuhelimen ohjelmistoarkkitehtuurille. Kolmannen sukupolven matkapuhelinjärjestelmät ovat aikaisempia järjestelmiä monimutkaisempia. Ohjelmiston koon ja monimutkaisuuden sekä aikataulujen kiireellisyyden vuoksi on tullut tarve ottaa käyttöön formaaleja menetelmiä ohjelmiston kehitystyöhön. Formaalit kuvauskielet mahdollistavat tarkan, yksiselitteisen ja simuloitavissa olevan järjestelmäkuvauksen muodostamisen. Fyysinen protokollakerros tarjoaa tiedon siirtoa ylemmille protokollakerroksille. Tämän tiedonsiirron hallinta vaatii protokollakerrosten välistä viestinvälitystä. Formaaleja kuvauskieliä käyttämällä voidaan viestinvälityksen toteutusta automatisoida ja siinä tarvittavaa logiikkaa havainnollistaa. Työssä suunniteltiin, toteutettiin ja testattiin ylempien protokollakerrosten kanssa kommunikoivaa osaa fyysisestä protokollakerroksesta. Tuloksena saatiin solunvalintatoiminnallisuuden vaatiman kommunikoinnin ja tilakoneen toteutus ohjelmistoarkkitehtuurissa. Ohjelmistonkehityksen alkuvaiheiden havaittiin olevan fyysisen kerroksen suorituskyvyn kannalta merkittävässä asemassa, koska tällöin viestinvälityksen optimointi on helpointa. Formaalit kuvauskielet eivät ole sellaisenaan täysin soveltuvia tarkoin määritellyn ohjelmistoarkkitehtuurin osien kehitykseen.

Fehlersuche im Modell - Modellbasiertes Testen und Debuggen

Kern der vorliegenden Arbeit ist die Erforschung von Methoden, Techniken und Werkzeugen zur Fehlersuche in modellbasierten Softwareentwicklungsprozessen. Hierzu wird zuerst ein von mir mitentwickelter, neuartiger und modellbasierter Softwareentwicklungsprozess, der sogenannte Fujaba Process, vorgestellt. Dieser Prozess wird von Usecase Szenarien getrieben, die durch spezielle Kollaborationsdiagramme formalisiert werden. Auch die weiteren Artefakte des Prozess bishin zur fertigen Applikation werden durch UML Diagrammarten modelliert. Es ist keine Programmierung im Quelltext nötig. Werkzeugunterstützung für den vorgestellte Prozess wird von dem Fujaba CASE Tool bereitgestellt. Große Teile der Werkzeugunterstützung für den Fujaba Process, darunter die Toolunterstützung für das Testen und Debuggen, wurden im Rahmen dieser Arbeit entwickelt. Im ersten Teil der Arbeit wird der Fujaba Process im Detail erklärt und unsere Erfahrungen mit dem Einsatz des Prozesses in Industrieprojekten sowie in der Lehre dargestellt. Der zweite Teil beschreibt die im Rahmen dieser Arbeit entwickelte Testgenerierung, die zu einem wichtigen Teil des Fujaba Process geworden ist. Hierbei werden aus den formalisierten Usecase Szenarien ausführbare Testfälle generiert. Es wird das zugrunde liegende Konzept, die konkrete technische Umsetzung und die Erfahrungen aus der Praxis mit der entwickelten Testgenerierung dargestellt. Der letzte Teil beschäftigt sich mit dem Debuggen im Fujaba Process. Es werden verschiedene im Rahmen dieser Arbeit entwickelte Konzepte und Techniken vorgestellt, die die Fehlersuche während der Applikationsentwicklung vereinfachen. Hierbei wurde darauf geachtet, dass das Debuggen, wie alle anderen Schritte im Fujaba Process, ausschließlich auf Modellebene passiert. Unter anderem werden Techniken zur schrittweisen Ausführung von Modellen, ein Objekt Browser und ein Debugger, der die rückwärtige Ausführung von Programmen erlaubt (back-in-time debugging), vorgestellt. Alle beschriebenen Konzepte wurden in dieser Arbeit als Plugins für die Eclipse Version von Fujaba, Fujaba4Eclipse, implementiert und erprobt. Bei der Implementierung der Plugins wurde auf eine enge Integration mit Fujaba zum einen und mit Eclipse auf der anderen Seite geachtet. Zusammenfassend wird also ein Entwicklungsprozess vorgestellt, die Möglichkeit in diesem mit automatischen Tests Fehler zu identifizieren und diese Fehler dann mittels spezieller Debuggingtechniken im Programm zu lokalisieren und schließlich zu beheben. Dabei läuft der komplette Prozess auf Modellebene ab. Für die Test- und Debuggingtechniken wurden in dieser Arbeit Plugins für Fujaba4Eclipse entwickelt, die den Entwickler bestmöglich bei der zugehörigen Tätigkeit unterstützen.

A transformational language for mutant description

Mutation testing has been used to assess the quality of test case suites by analyzing the ability in distinguishing the artifact under testing from a set of alternative artifacts, the so-called mutants. The mutants are generated from the artifact under testing by applying a set of mutant operators, which produce artifacts with simple syntactical differences. The mutant operators are usually based on typical errors that occur during the software development and can be related to a fault model. In this paper, we propose a language-named MuDeL (MUtant DEfinition Language)-for the definition of mutant operators, aiming not only at automating the mutant generation, but also at providing precision and formality to the operator definition. The proposed language is based on concepts from transformational and logical programming paradigms, as well as from context-free grammar theory. Denotational semantics formal framework is employed to define the semantics of the MuDeL language. We also describe a system-named mudelgen-developed to support the use of this language. An executable representation of the denotational semantics of the language is used to check the correctness of the implementation of mudelgen. At the very end, a mutant generator module is produced, which can be incorporated into a specific mutant tool/environment. (C) 2008 Elsevier Ltd. All rights reserved.

Improving data perturbation testing techniques for Web services

The widespread use of service-oriented architectures (SOAs) and Web services in commercial software requires the adoption of development techniques to ensure the quality of Web services. Testing techniques and tools concern quality and play a critical role in accomplishing quality of SOA based systems. Existing techniques and tools for traditional systems are not appropriate to these new systems, making the development of Web services testing techniques and tools required. This article presents new testing techniques to automatically generate a set of test cases and data for Web services. The techniques presented here explore data perturbation of Web services messages upon data types, integrity and consistency. To support these techniques, a tool (GenAutoWS) was developed and applied to real problems. (C) 2010 Elsevier Inc. All rights reserved.

Uma abordagem sistemática para implementação, gerenciamento e customização de testes de linhas de produto de software

Through the adoption of the software product line (SPL) approach, several benefits are achieved when compared to the conventional development processes that are based on creating a single software system at a time. The process of developing a SPL differs from traditional software construction, since it has two essential phases: the domain engineering - when common and variables elements of the SPL are defined and implemented; and the application engineering - when one or more applications (specific products) are derived from the reuse of artifacts created in the domain engineering. The test activity is also fundamental and aims to detect defects in the artifacts produced in SPL development. However, the characteristics of an SPL bring new challenges to this activity that must be considered. Several approaches have been recently proposed for the testing process of product lines, but they have been shown limited and have only provided general guidelines. In addition, there is also a lack of tools to support the variability management and customization of automated case tests for SPLs. In this context, this dissertation has the goal of proposing a systematic approach to software product line testing. The approach offers: (i) automated SPL test strategies to be applied in the domain and application engineering, (ii) explicit guidelines to support the implementation and reuse of automated test cases at the unit, integration and system levels in domain and application engineering; and (iii) tooling support for automating the variability management and customization of test cases. The approach is evaluated through its application in a software product line for web systems. The results of this work have shown that the proposed approach can help the developers to deal with the challenges imposed by the characteristics of SPLs during the testing process

Beta: uma ferramenta para geração de testes de unidade a partir de especificações B

Formal methods and software testing are tools to obtain and control software quality. When used together, they provide mechanisms for software specification, verification and error detection. Even though formal methods allow software to be mathematically verified, they are not enough to assure that a system is free of faults, thus, software testing techniques are necessary to complement the process of verification and validation of a system. Model Based Testing techniques allow tests to be generated from other software artifacts such as specifications and abstract models. Using formal specifications as basis for test creation, we can generate better quality tests, because these specifications are usually precise and free of ambiguity. Fernanda Souza (2009) proposed a method to define test cases from B Method specifications. This method used information from the machine s invariant and the operation s precondition to define positive and negative test cases for an operation, using equivalent class partitioning and boundary value analysis based techniques. However, the method proposed in 2009 was not automated and had conceptual deficiencies like, for instance, it did not fit in a well defined coverage criteria classification. We started our work with a case study that applied the method in an example of B specification from the industry. Based in this case study we ve obtained subsidies to improve it. In our work we evolved the proposed method, rewriting it and adding characteristics to make it compatible with a test classification used by the community. We also improved the method to support specifications structured in different components, to use information from the operation s behavior on the test case generation process and to use new coverage criterias. Besides, we have implemented a tool to automate the method and we have submitted it to more complex case studies

Uma ferramenta de análise automatizada de técnicas de seleção de testes de regressão baseada em mineração de repositórios de software

The main goal of Regression Test (RT) is to reuse the test suite of the latest version of a software in its current version, in order to maximize the value of the tests already developed and ensure that old features continue working after the new changes. Even with reuse, it is common that not all tests need to be executed again. Because of that, it is encouraged to use Regression Tests Selection (RTS) techniques, which aims to select from all tests, only those that reveal faults, this reduces costs and makes this an interesting practice for the testing teams. Several recent research works evaluate the quality of the selections performed by RTS techniques, identifying which one presents the best results, measured by metrics such as inclusion and precision. The RTS techniques should seek in the System Under Test (SUT) for tests that reveal faults. However, because this is a problem without a viable solution, they alternatively seek for tests that reveal changes, where faults may occur. Nevertheless, these changes may modify the execution flow of the algorithm itself, leading some tests no longer exercise the same stretch. In this context, this dissertation investigates whether changes performed in a SUT would affect the quality of the selection of tests performed by an RTS, if so, which features the changes present which cause errors, leading the RTS to include or exclude tests wrongly. For this purpose, a tool was developed using the Java language to automate the measurement of inclusion and precision averages achieved by a regression test selection technique for a particular feature of change. In order to validate this tool, an empirical study was conducted to evaluate the RTS technique Pythia, based on textual differencing, on a large web information system, analyzing the feature of types of tasks performed to evolve the SUT

Portable Hardness tester for timber classification

Hardness is a property largely used in material specifications, mechanical and metallurgical research and quality control of several materials. Specifically for timber, Janka hardness is a simple, quick and easy test, with good correlations with the compression parallel to grain strength, a strong reference in structural classification for this material. More recently, international studies have reported the use of Brinell hardness for timber assessment which resumes the advantages previously mentioned for Janka hardness and make it easier to be performed in the field, especially because of the lower magnitude of the involved loads. A first generation of an equipment for field evaluation of hardness in wood - Portable Hardness tester for wood - based on Brinell hardness has already been developed by the Research Group on Forest Products from FCA/UNESP, Brazil, with very good correlations between the evaluated hardness and several other mechanical properties of the material when performing tests with different species of native and reforested wood (traditionally used as ties - sleepers - in railways). This paper presents results obtained in the experimental program with the first generation of this equipment and preliminary tests with its second generation, which uses accelerometers to substitute the indentation measurements in wood. For the first generation of the equipment functional and calibration tests were carried out using 16 native and reforestation timber lots, among there E. citriodora, E. tereticornis, E. saligna, E. urophylla, E. grandis, Goupia glabra and Bagassa guianenses, with different origins and ages. The results obtained confirm its potential in the classification of specimens, with inclusion errors varying from 4.5% to 16.6%.

Designing Software Product Lines for Testability

Software product line (SPL) engineering offers several advantages in the development of families of software products such as reduced costs, high quality and a short time to market. A software product line is a set of software intensive systems, each of which shares a common core set of functionalities, but also differs from the other products through customization tailored to fit the needs of individual groups of customers. The differences between products within the family are well-understood and organized into a feature model that represents the variability of the SPL. Products can then be built by generating and composing features described in the feature model. Testing of software product lines has become a bottleneck in the SPL development lifecycle, since many of the techniques used in their testing have been borrowed from traditional software testing and do not directly take advantage of the similarities between products. This limits the overall gains that can be achieved in SPL engineering. Recent work proposed by both industry and the research community for improving SPL testing has begun to consider this problem, but there is still a need for better testing techniques that are tailored to SPL development. In this thesis, I make two primary contributions to software product line testing. First I propose a new definition for testability of SPLs that is based on the ability to re-use test cases between products without a loss of fault detection effectiveness. I build on this idea to identify elements of the feature model that contribute positively and/or negatively towards SPL testability. Second, I provide a graph based testing approach called the FIG Basis Path method that selects products and features for testing based on a feature dependency graph. This method should increase our ability to re-use results of test cases across successive products in the family and reduce testing effort. I report the results of a case study involving several non-trivial SPLs and show that for these objects, the FIG Basis Path method is as effective as testing all products, but requires us to test no more than 24% of the products in the SPL.

An integrated approach of software development and test processes to distributed teams

The Distributed Software Development (DSD) is a development strategy that meets the globalization needs concerned with the increase productivity and cost reduction. However, the temporal distance, geographical dispersion and the socio-cultural differences, increased some challenges and, especially, added new requirements related with the communication, coordination and control of projects. Among these new demands there is the necessity of a software process that provides adequate support to the distributed software development. This paper presents an integrated approach of software development and test that considers distributed teams peculiarities. The approach purpose is to offer support to DSD, providing a better project visibility, improving the communication between the development and test teams, minimizing the ambiguity and difficulty to understand the artifacts and activities. This integrated approach was conceived based on four pillars: (i) to identify the DSD peculiarities concerned with development and test processes, (ii) to define the necessary elements to compose the integrated approach of development and test to support the distributed teams, (iii) to describe and specify the workflows, artifacts, and roles of the approach, and (iv) to represent appropriately the approach to enable the effective communication and understanding of it.