Testing of Android testing tools: development of a benchmark for the evaluation

With the ever growing trend of smart phones and tablets, Android is becoming more and more popular everyday. With more than one billion active users i to date, Android is the leading technology in smart phone arena. In addition to that, Android also runs on Android TV, Android smart watches and cars. Therefore, in recent years, Android applications have become one of the major development sectors in software industry. As of mid 2013, the number of published applications on Google Play had exceeded one million and the cumulative number of downloads was more than 50 billionii. A 2013 survey also revealed that 71% of the mobile application developers work on developing Android applicationsiii. Considering this size of Android applications, it is quite evident that people rely on these applications on a daily basis for the completion of simple tasks like keeping track of weather to rather complex tasks like managing one’s bank accounts. Hence, like every other kind of code, Android code also needs to be verified in order to work properly and achieve a certain confidence level. Because of the gigantic size of the number of applications, it becomes really hard to manually test Android applications specially when it has to be verified for various versions of the OS and also, various device configurations such as different screen sizes and different hardware availability. Hence, recently there has been a lot of work on developing different testing methods for Android applications in Computer Science fraternity. The model of Android attracts researchers because of its open source nature. It makes the whole research model more streamlined when the code for both, application and the platform are readily available to analyze. And hence, there has been a great deal of research in testing and static analysis of Android applications. A great deal of this research has been focused on the input test generation for Android applications. Hence, there are a several testing tools available now, which focus on automatic generation of test cases for Android applications. These tools differ with one another on the basis of their strategies and heuristics used for this generation of test cases. But there is still very little work done on the comparison of these testing tools and the strategies they use. Recently, some research work has been carried outiv in this regard that compared the performance of various available tools with respect to their respective code coverage, fault detection, ability to work on multiple platforms and their ease of use. It was done, by running these tools on a total of 60 real world Android applications. The results of this research showed that although effective, these strategies being used by the tools, also face limitations and hence, have room for improvement. The purpose of this thesis is to extend this research into a more specific and attribute-­‐ oriented way. Attributes refer to the tasks that can be completed using the Android platform. It can be anything ranging from a basic system call for receiving an SMS to more complex tasks like sending the user to another application from the current one. The idea is to develop a benchmark for Android testing tools, which is based on the performance related to these attributes. This will allow the comparison of these tools with respect to these attributes. For example, if there is an application that plays some audio file, will the testing tool be able to generate a test input that will warrant the execution of this audio file? Using multiple applications using different attributes, it can be visualized that which testing tool is more useful for which kinds of attributes. In this thesis, it was decided that 9 attributes covering the basic nature of tasks, will be targeted for the assessment of three testing tools. Later this can be done for much more attributes to compare even more testing tools. The aim of this work is to show that this approach is effective and can be used on a much larger scale. One of the flagship features of this work, which also differentiates it with the previous work, is that the applications used, are all specially made for this research. The reason for doing that is to analyze just that specific attribute in isolation, which the application is focused on, and not allow the tool to get bottlenecked by something trivial, which is not the main attribute under testing. This means 9 applications, each focused on one specific attribute. The main contributions of this thesis are: A summary of the three existing testing tools and their respective techniques for automatic test input generation of Android Applications. • A detailed study of the usage of these testing tools using the 9 applications specially designed and developed for this study. • The analysis of the obtained results of the study carried out. And a comparison of the performance of the selected tools.

Dicto: A Unified DSL for Testing Architectural Rules

Software architecture consists of a set of design choices that can be partially expressed in form of rules that the implementation must conform to. Architectural rules are intended to ensure properties that fulfill fundamental non-functional requirements. Verifying architectural rules is often a non- trivial activity: available tools are often not very usable and support only a narrow subset of the rules that are commonly specified by practitioners. In this paper we present a new highly-readable declarative language for specifying architectural rules. With our approach, users can specify a wide variety of rules using a single uniform notation. Rules can get tested by third-party tools by conforming to pre-defined specification templates. Practitioners can take advantage of the capabilities of a growing number of testing tools without dealing with them directly.

A Framework for Design-Time Testing of Service-Based Applications at BPEL Level

Software applications created on top of the service-oriented architecture (SOA) are increasingly popular but testing them remains a challenge. In this paper a framework named TASSA for testing the functional and non-functional behaviour of service-based applications is presented. The paper focuses on the concept of design time testing, the corresponding testing approach and architectural integration of the consisting TASSA tools. The individual TASSA tools with sample validation scenarios were already presented with a general view of their relation. This paper’s contribution is the structured testing approach, based on the integral use of the tools and their architectural integration. The framework is based on SOA principles and is composable depending on user requirements.

A self -testing approach for autonomic software

As researchers and practitioners move towards a vision of software systems that configure, optimize, protect, and heal themselves, they must also consider the implications of such self-management activities on software reliability. Autonomic computing (AC) describes a new generation of software systems that are characterized by dynamically adaptive self-management features. During dynamic adaptation, autonomic systems modify their own structure and/or behavior in response to environmental changes. Adaptation can result in new system configurations and capabilities, which need to be validated at runtime to prevent costly system failures. However, although the pioneers of AC recognize that validating autonomic systems is critical to the success of the paradigm, the architectural blueprint for AC does not provide a workflow or supporting design models for runtime testing. ^ This dissertation presents a novel approach for seamlessly integrating runtime testing into autonomic software. The approach introduces an implicit self-test feature into autonomic software by tailoring the existing self-management infrastructure to runtime testing. Autonomic self-testing facilitates activities such as test execution, code coverage analysis, timed test performance, and post-test evaluation. In addition, the approach is supported by automated testing tools, and a detailed design methodology. A case study that incorporates self-testing into three autonomic applications is also presented. The findings of the study reveal that autonomic self-testing provides a flexible approach for building safe, reliable autonomic software, while limiting the development and performance overhead through software reuse. ^

一种面向开源测试工具的集成框架的设计与实现

在软件测试过程中，优秀的测试工具通常能显著地提高测试的效率。在开源社区的推动下，越来越多的开源测试工具不断涌现。软件测试是一个复杂的过程，而现有的开源测试工具大多专注于测试过程的一个方面，如静态代码分析、单元测试、测试管理乃至性能测试。要完成一个完整的测试流程，往往需要结合使用多种测试工具。这些工具彼此独立，相互间缺乏沟通的标准和途径，导致需要付出大量人工劳动来完成信息的传递，以至于整个测试过程效率的降低和成本的提高。为了提高测试相关工具间交互的效率，人们往往采用硬编码等方式将工具进行集成。然而，旧有的集成方式使得工具间耦合度大大提高，代码复用性差，缺乏扩展性，并不适合开源测试工具种类多、涉及面广的特点。 为了在提高开源测试相关工具间协作效率的同时，保持一定的复用性与扩展性，本文提出并实现了一种面向开源测试工具的集成框架，提供了一个供不同测试工具进行信息的传递和共享的平台。该框架将各种测试工具的输入输出接口提取出来，用一个可扩展的接口定义将它们统一，使各个工具之间可以在互相没有察觉的情况下进行测试数据的传递，降低了工具间的耦合性。统一接口可扩展的特点使得框架能适应不同测试工具的信息接口，提高了系统的灵活性。 本文介绍了开源测试工具在当前测试工作中的应用以及在工具协作中存在的问题，分析了对开源测试工具进行集成的需求，并介绍了常见的工具集成方法。基于其中的框架式集成思想，设计并实现了一个面向开源测试工具的集成框架，详细介绍了该框架的设计思想和实现技术。在此基础上，通过将几个开源测试工具与框架进行集成并完成相互间的交互验证了框架的可用性。

Simulation of coupled heat and moisture transfer in air-conditioned buildings

The simultaneous heat and moisture transfer in the building envelope has an important influence on the indoor environment and the overall performance of buildings. In this paper, a model for predicting whole building heat and moisture transfer was presented. Both heat and moisture transfer in the building envelope and indoor air were simultaneously considered; their interactions were modeled. The coupled model takes into account most of the main hygrothermal effects in buildings. The coupled system model was implemented in MATLAB-Simulink, and validated by using a series of published testing tools. The new program was applied to investigate the moisture transfer effect on indoor air humidity and building energy consumption under different climates. The results show that the use of more detailed simulation routines can result in improvements to the building's design for energy optimisation through the choice of proper hygroscopic materials, which would not be indicated by simpler calculation techniques.

Simulation of whole building coupled hygrothermal-airflow transfer in different climates

The coupled heat, air and moisture transfer between building envelopes and indoor air is complicated, and has a significant influence on the indoor environment and the energy performance of buildings. In the paper, a model for predicting coupled multi-zone hygrothermal-airflow transfer is presented. Both heat and moisture transfer in the building envelope and multi-zone indoor airflow are simultaneously considered; their interactions are modeled. The coupled system model is implemented into Matlab–Simulink, and is validated by using a series of testing tools and experiments. The new program is applied to investigate the moisture transfer effect on indoor air humidity and building energy consumption in different climates (hot-humid, temperate and hot-dry climates). The results show that not accounting for hygrothermal effects in modeling will result in overestimation of energy costs for hot and humid climate situations and possible over sizing of plant leading to inefficient operation.

Testverktyg för test av mobila applikationer : En jämförande fallstudie

Användandet av mobila applikationer har växt radikalt de senaste åren och de samverkar med många system. Därför ställs det högre krav på kvaliteten och att applikationen ska anpassas till många olika enheter, operativsystem samt plattformar. Detta gör att test av mobila applikationer blivit viktigare och större. Detta arbete har bedrivits som en jämförande fallstudie inom området test av mobila applikationer samt testverktyg. Syftet har varit att beskriva hur testning av mobila applikationer sker idag vilket gjorts genom litteraturstudier och intervjuer med IT-företag. Ett annat syfte har varit att utvärdera fyra testverktyg, deras för- och nackdelar samt hur de kan användas vid testning av mobila applikationer och jämföras mot manuell testning utan testverktyg. Detta har gjorts genom att skapa förstahandserfarenheter baserat på användandet av testverktygen. Under arbetet har vi utgått från mobila applikationer som vi fått tillgång till av Triona, som varit vår samarbetspartner.Idag finns många olika testverktyg som kan användas som stöd för testningen men få företag har implementerat något eftersom det kräver både tid och kompetens samt valet av testverktyg kan vara svårt. Testverktygen har olika för- och nackdelar vilket gör att de passar olika bra beroende på typ av projekt och applikation. Fördelar med att använda testverktyg är möjligheten att kunna automatisera, testa på flera enheter samtidigt samt få tillgång till enheter via molnet. Utmaningarna är att det kan vara svårt att installera och lära sig testverktyget samt att licenserna kan vara dyra. Det är därför viktigt att redan innan implementationen veta vilka tester och applikationer testverktygen ska användas till samt vem som ska använda det. Utifrån vår studie kan slutsatsen dras att inget testverktyg är helt komplett men de kan bidra med olika funktioner vilket effektiviserar delar av testningen av mobila applikationer.

Clinical association between pharmacogenomics and adverse drug reactions

Adverse drug reactions (ADRs) are a major public health concern and cause significant patient morbidity and mortality. Pharmacogenomics is the study of how genetic polymorphisms affect an individual’s response to pharmacotherapy at the level of a whole genome. This article updates our knowledge on how genetic polymorphisms of important genes alter the risk of ADR occurrence after an extensive literature search. To date, at least 244 pharmacogenes identified have been associated with ADRs of 176 clinically used drugs based on PharmGKB. At least 28 genes associated with the risk of ADRs have been listed by the Food and Drug Administration as pharmacogenomic biomarkers. With the availability of affordable and reliable testing tools, pharmacogenomics looks promising to predict, reduce, and minimize ADRs in selected populations.

A preliminary survey of factors affecting software testers

Most software testing research has focused on the development of systematic, standardised, and automated testing methodologies and tools. The abilities and expertise needed to apply such techniques and tools - such as personality traits, education, and experience - have attracted a comparatively small amount of research attention. However, the limited research in the area to date provides some indication that the human traits of software testers are important for effective testing. This paper presents the opinions of software testers themselves, collected through an online survey, on the importance of a variety of factors that influence effective testing, including testing-specific training, experience, skills, and human qualities like dedication and general intelligence. The survey responses strongly suggest that while testing tools and training are important, human factors were similarly considered highly important. Domain knowledge, experience, intelligence, and dedication, amongst other traits, were considered crucial for a software tester to be effective. As such, while systematic methodologies are important, the individual most clearly does matter in software testing. The results of our research have implications for education, recruitment, training and management of software testers.

The resistivity piezocone for high resolution geo-environmental site investigation: A Brazilian example

A suit able decision-making on managing a contaminated site characterization program is strongly dependent of the diagnosis process. A detailed diagnosis can be done based on a Conceptual Site Model (CSM) elaboration using high resolution site characterization tools. The piezocone (CPTu) test is a high resolution tool which allows attaching several specific sensors, like the resistivity probe. This hybrid device is called the resistivity piezocone (RCPTu). A simulated geo-environmental site characterization program was performed on an erosion site using different tools (direct push tools soil samplers, hollow stem auger (HSA) drilling and RCPTu tests) to develop the CSM for a site similar to the Brazilian conditions. It was observed a good agreement between the site profiles interpreted by the different methods. The resistivity sensor attached to the piezocone improved the interpretation and the decision-making process on site was significantly better for the CSM elaboration. The RCPTu test data also allowed identifying the hydrogeological heterogeneities. The present study shows that the RCPTu test is also a useful and powerful tool to development an accurate CSM in a Brazilian condition, especially in an approach that prioritizes high resolution geo-environmental investigation. © 2013 Taylor & Francis Group.

THE DESIGN OF A POWER SYSTEM FOR A WIRELESS SENSOR NETWORK FOR LONG-TERM OPERATION

Wireless sensor networks (WSNs) have shown wide applicability to many fields including monitoring of environmental, civil, and industrial settings. WSNs however are resource constrained by many competing factors that span their hardware, software, and networking. One of the central resource constrains is the charge consumption of WSN nodes. With finite energy supplies, low charge consumption is needed to ensure long lifetimes and success of WSNs. This thesis details the design of a power system to support long-term operation of WSNs. The power system’s development occurs in parallel with a custom WSN from the Queen’s MEMS Lab (QML-WSN), with the goal of supporting a 1+ year lifetime without sacrificing functionality. The final power system design utilizes a TPS62740 DC-DC converter with AA alkaline batteries to efficiently supply the nodes while providing battery monitoring functionality and an expansion slot for future development. Testing tools for measuring current draw and charge consumption were created along with analysis and processing software. Through their use charge consumption of the power system was drastically lowered and issues in QML-WSN were identified and resolved including the proper shutdown of accelerometers, and incorrect microcontroller unit (MCU) power pin connection. Controlled current profiling revealed unexpected behaviour of nodes and detailed current-voltage relationships. These relationships were utilized with a lifetime projection model to estimate a lifetime between 521-551 days, depending on the mode of operation. The power system and QML-WSN were tested over a long term trial lasting 272+ days in an industrial testbed to monitor an air compressor pump. Environmental factors were found to influence the behaviour of nodes leading to increased charge consumption, while a node in an office setting was still operating at the conclusion of the trail. This agrees with the lifetime projection and gives a strong indication that a 1+ year lifetime is achievable. Additionally, a light-weight charge consumption model was developed which allows charge consumption information of nodes in a distributed WSN to be monitored. This model was tested in a laboratory setting demonstrating +95% accuracy for high packet reception rate WSNs across varying data rates, battery supply capacities, and runtimes up to full battery depletion.

THE DESIGN OF A POWER SYSTEM FOR A WIRELESS SENSOR NETWORK FOR LONG-TERM OPERATION

Wireless sensor networks (WSNs) have shown wide applicability to many fields including monitoring of environmental, civil, and industrial settings. WSNs however are resource constrained by many competing factors that span their hardware, software, and networking. One of the central resource constrains is the charge consumption of WSN nodes. With finite energy supplies, low charge consumption is needed to ensure long lifetimes and success of WSNs. This thesis details the design of a power system to support long-term operation of WSNs. The power system’s development occurs in parallel with a custom WSN from the Queen’s MEMS Lab (QML-WSN), with the goal of supporting a 1+ year lifetime without sacrificing functionality. The final power system design utilizes a TPS62740 DC-DC converter with AA alkaline batteries to efficiently supply the nodes while providing battery monitoring functionality and an expansion slot for future development. Testing tools for measuring current draw and charge consumption were created along with analysis and processing software. Through their use charge consumption of the power system was drastically lowered and issues in QML-WSN were identified and resolved including the proper shutdown of accelerometers, and incorrect microcontroller unit (MCU) power pin connection. Controlled current profiling revealed unexpected behaviour of nodes and detailed current-voltage relationships. These relationships were utilized with a lifetime projection model to estimate a lifetime between 521-551 days, depending on the mode of operation. The power system and QML-WSN were tested over a long term trial lasting 272+ days in an industrial testbed to monitor an air compressor pump. Environmental factors were found to influence the behaviour of nodes leading to increased charge consumption, while a node in an office setting was still operating at the conclusion of the trail. This agrees with the lifetime projection and gives a strong indication that a 1+ year lifetime is achievable. Additionally, a light-weight charge consumption model was developed which allows charge consumption information of nodes in a distributed WSN to be monitored. This model was tested in a laboratory setting demonstrating +95% accuracy for high packet reception rate WSNs across varying data rates, battery supply capacities, and runtimes up to full battery depletion.

Fault diagnostics and supervised testing : How fault diagnostic tools can be proactive?

The topic of fault detection and diagnostics (FDD) is studied from the perspective of proactive testing. Unlike most research focus in the diagnosis area in which system outputs are analyzed for diagnosis purposes, in this paper the focus is on the other side of the problem: manipulating system inputs for better diagnosis reasoning. In other words, the question of how diagnostic mechanisms can direct system inputs for better diagnosis analysis is addressed here. It is shown how the problem can be formulated as decision making problem coupled with a Bayesian Network based diagnostic mechanism. The developed mechanism is applied to the problem of supervised testing in HVAC systems.