Empirical study on the adoption, use and effects of cloud-based testing

Cloud computing is a practically relevant paradigm in computing today. Testing is one of the distinct areas where cloud computing can be applied. This study addressed the applicability of cloud computing for testing within organizational and strategic contexts. The study focused on issues related to the adoption, use and effects of cloudbased testing. The study applied empirical research methods. The data was collected through interviews with practitioners from 30 organizations and was analysed using the grounded theory method. The research process consisted of four phases. The first phase studied the definitions and perceptions related to cloud-based testing. The second phase observed cloud-based testing in real-life practice. The third phase analysed quality in the context of cloud application development. The fourth phase studied the applicability of cloud computing in the gaming industry. The results showed that cloud computing is relevant and applicable for testing and application development, as well as other areas, e.g., game development. The research identified the benefits, challenges, requirements and effects of cloud-based testing; and formulated a roadmap and strategy for adopting cloud-based testing. The study also explored quality issues in cloud application development. As a special case, the research included a study on applicability of cloud computing in game development. The results can be used by companies to enhance the processes for managing cloudbased testing, evaluating practical cloud-based testing work and assessing the appropriateness of cloud-based testing for specific testing needs.

Cost-benefit analysis of using test automation in the development of embedded software

The goal of this thesis is to make a case study of test automation’s profitability in the development of embedded software in a real industrial setting. The cost-benefit analysis is done by considering the costs and benefits test automation causes to software development, before the software is released to customers. The potential benefits of test automation regarding software quality after customer release were not estimated. Test automation is a significant investment which often requires dedicated resources. When done accordingly, the investment in test automation can produce major cost savings by reducing the need for manual testing effort, especially if the software is developed with an agile development framework. It can reduce the cost of avoidable rework of software development, as test automation enables the detection of construction time defects in the earliest possible moment. Test automation also has many pitfalls such as test maintainability and testability of the software, and if those areas are neglected, the investment in test automation may become worthless or it may even produce negative results. The results of this thesis suggest that test automation is very profitable at the company under study.

Software development methods and quality assurance: Special focus on South Korea

The purpose of this study was to explore software development methods and quality assurance practices used by South Korean software industry. Empirical data was collected by conducting a survey that focused on three main parts: software life cycle models and methods, software quality assurance including quality standards, the strengths and weaknesses of South Korean software industry. The results of the completed survey showed that the use of agile methods is slightly surpassing the use of traditional software development methods. The survey also revealed an interesting result that almost half of the South Korean companies do not use any software quality assurance plan in their projects. For the state of South Korean software industry large number of the respondents thought that despite of the weakness, the status of software development in South Korea will improve in the future.

Empirical studies on software quality construction: Exploring human factors and organizational influences

Software quality has become an important research subject, not only in the Information and Communication Technology spheres, but also in other industries at large where software is applied. Software quality is not a happenstance; it is defined, planned and created into the software product throughout the Software Development Life Cycle. The research objective of this study is to investigate the roles of human and organizational factors that influence software quality construction. The study employs the Straussian grounded theory. The empirical data has been collected from 13 software companies, and the data includes 40 interviews. The results of the study suggest that tools, infrastructure and other resources have a positive impact on software quality, but human factors involved in the software development processes will determine the quality of the products developed. On the other hand, methods of development were found to bring little effect on software quality. The research suggests that software quality is an information-intensive process whereby organizational structures, mode of operation, and information flow within the company variably affect software quality. The results also suggest that software development managers influence the productivity of developers and the quality of the software products. Several challenges of software testing that affect software quality are also brought to light. The findings of this research are expected to benefit the academic community and software practitioners by providing an insight into the issues pertaining to software quality construction undertakings.

The functional fit between collaborative software and work systems:Qualification of work system needs to software functionality

The study develops an approach that tries to validate software functionality to work systems needs in SMEs. The formulated approach is constructed by using a SAAS based software i.e., work collaboration service (WCS), and SMEs as the elements of study. Where the WCS’s functionality is qualified to the collaboration needs that exist in operational and project work within SMEs. For this research constructivist approach and case study method is selected because the nature of the current study requires an in depth study of the work collaboration service as well as a detailed study of the work systems within different enterprises. Four different companies are selected in which fourteen interviews are conducted to gather data pertaining. The work systems method and framework are used as a central part of the approach to collect, analyze and interpret the enterprises work systems model and the underlying collaboration needs on operational and project work. On the other hand, the functional model of the WCS and its functionality is determined from functional model analysis, software testing, documentation and meetings with the service vendor. The enterprise work system model and the WCS model are compared to reveal how work progression differs between the two and make visible unaddressed stages of work progression. The WCS functionality is compared to work systems collaboration needs to ascertain if the service will suffice the needs of the project and operational work under study. The unaddressed needs provide opportunities to improve the functionality of the service for better conformity to the needs of enterprise and work. The results revealed that the functional models actually differed in how operational and project work progressed within the stages. WCS shared similar stages of work progression apart from the stages of identification and acceptance, and progress and completion stages were only partially addressed. Conclusion is that the identified unaddressed needs such as, single point of reference, SLA and OLA inclusion etc., should be implemented or improved within the WCS at appropriate stages of work to gain better compliance of the service to the needs of the enterprise an work itself. The developed approach can hence be used to carry out similar analysis for the conformance of pre-built software functionality to work system needs with SMEs.

Virtuaalisen oppimisympäristön funktionaalisen käyttöliittymätestauksen automatisointi

Tämän tutkimuksen kohteena on Turun yliopistossa kehitetyn www-pohjaisen ViLLE-oppimisjärjestelmän funktionaalisen käyttöliittymätestauksen automatisointityö. Tutkimusta varten olen kerännyt kattavasti aineistoa aihetta käsittelevästä yleisestä kirjallisuudesta ja artikkeleista sekä toteutuksen kannalta spesifistä tietoa tarjoavista Internet-lähteistä. Tutkimuksessa olen tehnyt myös pienehkön määrän testausalan asiantuntijahaastatteluja. Tutkimuksen empiirisessä osuudessa olen valinnut testaukseen käytettävän testaustyökalun sekä toteuttanut valitulla testaustyökalulla ViLLE-oppimisjärjestelmän testauksen automatisointityön soveltamalla käytäntöön tutkimuksen teoriaosuudessa esitettyä tietoa hyvistä käytänteistä funktionaalisen käyttöliittymätestauksen automatisoinnissa. Tutkimuksen toteutuksessa olen käyttänyt kvalitatiivista tutkimusmenetelmää. Tutkimuksen empiirisen osuuden pohjalta kerätyn havaintoaineiston perusteella olen selvittänyt vastaukset seuraaviin tutkimuksessa esitettyihin tutkimuskysymyksiin: • Miten käytetty testaustyökalu on valittu ja mitkä olivat valintaan vaikuttaneet tärkeimmät kriteerit? • Miten käytetty testaustyökalu soveltuu ViLLE-oppimisjärjestelmän funktionaalisen käyttöliittymätestauksen automatisointiin? • Millä eri tavoin käytäntöön viety hyvien testiautomaation laatimistapojen mukainen toteutus vaikuttaa nyt laadittuun testiautomaatioon? • Esiintyikö toteutetussa testiautomaatiossa tutkimuksen teoreettisessa viitekehyksessä kuvattuja funktionaalisen käyttöliittymätestauksen automatisoinnille tyypillisiä ongelmia ja miten ongelmat saatiin ratkaistua? Tutkimuksen tulokset osoittavat melko selvästi, että ViLLE-oppimisympäristön funktionaalisen käyttöliittymätestauksen automatisointityön toteutukseen valittu Vaadin TestBench -testaustyökalu, joka on valittu tutkimuksen alkuvaiheessa suoritetun evaluoinnin perusteella, soveltuu käyttötarkoitukseensa hyvin. Lisäksi pystyin luotettavasti havainnoimaan, että testiautomaation ylläpidon tarve sekä testien laatimiseen kuluva aika vähenevät merkittävästi, kun testit laaditaan heti alusta lähtien rakenteeltaan modulaariseksi sekä tietyin teknisin keinoin mahdollisimman vähän käyttöliittymän rakennetta huomioonottavaksi. Ongelmia testiautomaation laatimisessa voivat aiheuttaa käytetty työkalu itsessään, testattavan järjestelmän toteutus sekä testien suoritusympäristö. Huolimatta kirjallisuuskatsauksen perusteella tehdystä varautumisesta tyypillisiin testiautomaation laatimisessa esiintyviin ongelmiin, myös joitakin sellaisia ongelmia esiintyi, joihin en ollut osannut varautua. Mahdollisiin ongelmiin etukäteen varautuminen kuitenkin selvästi auttoi suurimpaan osaan testiautomaation laatimisessa esiintyneistä ongelmista.

The automation and evaluation of nested clade phylogeographic analysis

Nested clade phylogeographic analysis (NCPA) is a popular method for reconstructing the demographic history of spatially distributed populations from genetic data. Although some parts of the analysis are automated, there is no unique and widely followed algorithm for doing this in its entirety, beginning with the data, and ending with the inferences drawn from the data. This article describes a method that automates NCPA, thereby providing a framework for replicating analyses in an objective way. To do so, a number of decisions need to be made so that the automated implementation is representative of previous analyses. We review how the NCPA procedure has evolved since its inception and conclude that there is scope for some variability in the manual application of NCPA. We apply the automated software to three published datasets previously analyzed manually and replicate many details of the manual analyses, suggesting that the current algorithm is representative of how a typical user will perform NCPA. We simulate a large number of replicate datasets for geographically distributed, but entirely random-mating, populations. These are then analyzed using the automated NCPA algorithm. Results indicate that NCPA tends to give a high frequency of false positives. In our simulations we observe that 14% of the clades give a conclusive inference that a demographic event has occurred, and that 75% of the datasets have at least one clade that gives such an inference. This is mainly due to the generation of multiple statistics per clade, of which only one is required to be significant to apply the inference key. We survey the inferences that have been made in recent publications and show that the most commonly inferred processes (restricted gene flow with isolation by distance and contiguous range expansion) are those that are commonly inferred in our simulations. However, published datasets typically yield a richer set of inferences with NCPA than obtained in our random-mating simulations, and further testing of NCPA with models of structured populations is necessary to examine its accuracy.

An aspect-oriented reference architecture for Software Engineering Environments

Reusable and evolvable Software Engineering Environments (SEES) are essential to software production and have increasingly become a need. In another perspective, software architectures and reference architectures have played a significant role in determining the success of software systems. In this paper we present a reference architecture for SEEs, named RefASSET, which is based on concepts coming from the aspect-oriented approach. This architecture is specialized to the software testing domain and the development of tools for that domain is discussed. This and other case studies have pointed out that the use of aspects in RefASSET provides a better Separation of Concerns, resulting in reusable and evolvable SEEs. (C) 2011 Elsevier Inc. All rights reserved.

Improving data perturbation testing techniques for Web services

The widespread use of service-oriented architectures (SOAs) and Web services in commercial software requires the adoption of development techniques to ensure the quality of Web services. Testing techniques and tools concern quality and play a critical role in accomplishing quality of SOA based systems. Existing techniques and tools for traditional systems are not appropriate to these new systems, making the development of Web services testing techniques and tools required. This article presents new testing techniques to automatically generate a set of test cases and data for Web services. The techniques presented here explore data perturbation of Web services messages upon data types, integrity and consistency. To support these techniques, a tool (GenAutoWS) was developed and applied to real problems. (C) 2010 Elsevier Inc. All rights reserved.