A Binary Classifier for Test Case Feasibility Applied to Automatically Generated Tests of Event-Driven Software

Modern software application testing, such as the testing of software driven by graphical user interfaces (GUIs) or leveraging event-driven architectures in general, requires paying careful attention to context. Model-based testing (MBT) approaches first acquire a model of an application, then use the model to construct test cases covering relevant contexts. A major shortcoming of state-of-the-art automated model-based testing is that many test cases proposed by the model are not actually executable. These \textit{infeasible} test cases threaten the integrity of the entire model-based suite, and any coverage of contexts the suite aims to provide. In this research, I develop and evaluate a novel approach for classifying the feasibility of test cases. I identify a set of pertinent features for the classifier, and develop novel methods for extracting these features from the outputs of MBT tools. I use a supervised logistic regression approach to obtain a model of test case feasibility from a randomly selected training suite of test cases. I evaluate this approach with a set of experiments. The outcomes of this investigation are as follows: I confirm that infeasibility is prevalent in MBT, even for test suites designed to cover a relatively small number of unique contexts. I confirm that the frequency of infeasibility varies widely across applications. I develop and train a binary classifier for feasibility with average overall error, false positive, and false negative rates under 5\%. I find that unique event IDs are key features of the feasibility classifier, while model-specific event types are not. I construct three types of features from the event IDs associated with test cases, and evaluate the relative effectiveness of each within the classifier. To support this study, I also develop a number of tools and infrastructure components for scalable execution of automated jobs, which use state-of-the-art container and continuous integration technologies to enable parallel test execution and the persistence of all experimental artifacts.

一种基于安全状态转移的简并测试集生成方法

在总结前人工作的基础上,结合安全操作系统对测试的特殊需求,提出了简并测试集(degenerate test set,简称DTS)的概念,设计了一种使用模型检测的基于安全状态转移的高效测试集生成方法.该方法以状态转移为化简对象,在利用模型检测技术生成测试用例的同时,归并相同的状态转移并化简需求集中的冗余属性,从而最终达到化简测试集的目的.在此基础上,探讨了单个用例失败时用例集的有效性问题,并对DTS生成算法进行了改进.实验结果表明,该方法可以有效地对测试集中的冗余进行化简.

The design of multi-element airfoils through multi-objective optimization techniques

This paper presents the development and the application of a multi-objective optimization framework for the design of two-dimensional multi-element high-lift airfoils. An innovative and efficient optimization algorithm, namely Multi-Objective Tabu Search (MOTS), has been selected as core of the framework. The flow-field around the multi-element configuration is simulated using the commercial computational fluid dynamics (cfd) suite Ansys cfx. Elements shape and deployment settings have been considered as design variables in the optimization of the Garteur A310 airfoil, as presented here. A validation and verification process of the cfd simulation for the Garteur airfoil is performed using available wind tunnel data. Two design examples are presented in this study: a single-point optimization aiming at concurrently increasing the lift and drag performance of the test case at a fixed angle of attack and a multi-point optimization. The latter aims at introducing operational robustness and off-design performance into the design process. Finally, the performance of the MOTS algorithm is assessed by comparison with the leading NSGA-II (Non-dominated Sorting Genetic Algorithm) optimization strategy. An equivalent framework developed by the authors within the industrial sponsor environment is used for the comparison. To eliminate cfd solver dependencies three optimum solutions from the Pareto optimal set have been cross-validated. As a result of this study MOTS has been demonstrated to be an efficient and effective algorithm for aerodynamic optimizations. Copyright © 2012 Tech Science Press.

Computing complete test graphs for hierarchical systems

Conformance testing focuses on checking whether an implementation. under test (IUT) behaves according to its specification. Typically, testers are interested it? performing targeted tests that exercise certain features of the IUT This intention is formalized as a test purpose. The tester needs a "strategy" to reach the goal specified by the test purpose. Also, for a particular test case, the strategy should tell the tester whether the IUT has passed, failed. or deviated front the test purpose. In [8] Jeron and Morel show how to compute, for a given finite state machine specification and a test purpose automaton, a complete test graph (CTG) which represents all test strategies. In this paper; we consider the case when the specification is a hierarchical state machine and show how to compute a hierarchical CTG which preserves the hierarchical structure of the specification. We also propose an algorithm for an online test oracle which avoids a space overhead associated with the CTG.

一种编译优化测试用例自动生成方法的设计与实现

编译器的质量保证对提高软件产品的质量有着重要作用,对编译优化的测试是其中的核心部分.对编译优化的测试需要大量的测试用例程序.要构造这些测试用例,使用传统手工构造方法面临着效率低的问题,而基于文法的构造方法则针对性不足.从对优化的形式化描述出发来自动构造测试用例能克服这些缺点.本文设计并实现了一种基于形式化描述的编译优化测试用例程序生成方法.该方法基于编译优化的时序逻辑描述构造关键顶点控制流图,逐步转换为控制流图并得到用例程序.针对GCC(版本4.1.1)进行的覆盖率测试实验表明,该方法可以生成具有较高针对性的测试用例,并达到相当的覆盖程度.

基于时序逻辑的编译优化测试用例生成方法

提出一种基于时序逻辑公式的关键节点控制图生成方法,生成的测试用例针对性强,容易扩展;并以该方法改进了一种编译优化自动化测试工具,在很大程度上消除了其测试冗余,提高了测试效率

基于分支时序逻辑的编译优化测试用例自动生成方法的研究与实现

编译器是软件产业中重要的工具，对它的质量保证非常重要。编译优化是编译器的重要功能，它的质量对于编译器质量有重大影响。可采用软件测试的方法进行编译器优化模块的质量保证。测试需要测试用例。编译优化的测试用例必须触发编译器的优化功能，是具有可被优化特征的源程序。对不同的编译优化，该特征各不相同。需要将不同优化所对应的特征加入到源程序中以构造编译优化测试用例程序。 TRANS语言结合了时序逻辑，描述了不同的编译优化，包括优化前后的代码特征、优化执行的条件及方法。优化前的代码特征和执行优化的条件可被用作构造编译优化测试用例程序所需的特征。一种基于时序逻辑的编译优化测试用例程序生成方法的框架已被提出。该方法从TRANS描述的某种变体生成编译优化测试用例程序。但是该框架并未完善，面临多方面的问题。本文参考该框架的思想，设计了编译优化测试用例程序生成方法，解决了算法框架的部分问题。该方法可以适应复杂描述的情况；公式的合法性及语义得以保持；具体化并完整化了原有框架。该方法是具有针对性的编译优化测试用例程序自动生成方法。本文对该方法作了原型系统实现，并从中得到测试用例程序。本文设计并进行针对GCC的优化模块测试实验，以覆盖率为评价指标检验了测试用例程序的质量。实验表明该方法生成的测试用例程序具有针对性。对编译优化模块的测试，该方法是一种行之有效的办法。并且该方法仍有更多的应用空间，加以改进后可用于优化组合测试、优化正确性检测等。

Optimal dispatch with reactive power compensation by Genetic Algorithm

This paper presents a Unit Commitment model with reactive power compensation that has been solved by Genetic Algorithm (GA) optimization techniques. The GA has been developed a computational tools programmed/coded in MATLAB. The main objective is to find the best generations scheduling whose active power losses are minimal and the reactive power to be compensated, subjected to the power system technical constraints. Those are: full AC power flow equations, active and reactive power generation constraints. All constraints that have been represented in the objective function are weighted with a penalty factors. The IEEE 14-bus system has been used as test case to demonstrate the effectiveness of the proposed algorithm. Results and conclusions are dully drawn.

Assessing species’ habitat associations from occurrence records, standardised monitoring data and expert opinion: a test with British butterflies

Accurate knowledge of species’ habitat associations is important for conservation planning and policy. Assessing habitat associations is a vital precursor to selecting appropriate indicator species for prioritising sites for conservation or assessing trends in habitat quality. However, much existing knowledge is based on qualitative expert opinion or local scale studies, and may not remain accurate across different spatial scales or geographic locations. Data from biological recording schemes have the potential to provide objective measures of habitat association, with the ability to account for spatial variation. We used data on 50 British butterfly species as a test case to investigate the correspondence of data-derived measures of habitat association with expert opinion, from two different butterfly recording schemes. One scheme collected large quantities of occurrence data (c. 3 million records) and the other, lower quantities of standardised monitoring data (c. 1400 sites). We used general linear mixed effects models to derive scores of association with broad-leaf woodland for both datasets and compared them with scores canvassed from experts. Scores derived from occurrence and abundance data both showed strongly positive correlations with expert opinion. However, only for occurrence data did these fell within the range of correlations between experts. Data-derived scores showed regional spatial variation in the strength of butterfly associations with broad-leaf woodland, with a significant latitudinal trend in 26% of species. Sub-sampling of the data suggested a mean sample size of 5000 occurrence records per species to gain an accurate estimation of habitat association, although habitat specialists are likely to be readily detected using several hundred records. Occurrence data from recording schemes can thus provide easily obtained, objective, quantitative measures of habitat association.

En beskrivning av manuellt test : Svagheter och styrkor med och utan stöd avett testverktyg

Test is an area in system development. Test can be performed manually or automated. Test activities can be supported by Word documents and Excel sheets for documenting and executing test cases and as well for follow up, but there are also new test tools designed to support and facilitate the testing process and the activities of the test. This study has described manual test and identified strengths and weaknesses of manual testing with a testing tool called Microsoft Test Manager (MTM) and of manual testing using test cases and test log templates developed by the testers at Sogeti. The result that emerged from the problem and strength analysis and the analysis of literature studies and firsthand experiences (in terms of creating, documenting and executing test cases) addresses the issue of the following weaknesses and strengths. Strengths of the test tool is that it contains needed functionality all in one place and it is available when needed without having to open up other programs which saves many steps of activity. Strengths with test without the support of test tools is mainly that it is easy to learn and gives a good overview, easy to format text as desired and flexible to changes during execution of a test case. Weaknesses in test with the support of test tools include that it is difficult to get a good overview of the entire test case, that it is not possible to format the text in the test steps. It is as well not possible to modify the test steps during execution. It is also difficult to use some of the test design techniques of TMap, for example a checklist, when using the test tool MTM. Weaknesses with test without the support of the testing tool MTM is that the tester gets many more steps of activities to do compared to doing the same activities with the support of the testing tool MTM. There is more to remember because the documents the tester use are not directly linked. Altogether the strengths of the test tool stands out when it comes to supporting the testing process.

Evolutionary Testing of Autonomous Software Agents

A system built in terms of autonomous agents may require even greater correctness assurance than one which is merely reacting to the immediate control of its users. Agents make substantial decisions for themselves, so thorough testing is an important consideration. However, autonomy also makes testing harder; by their nature, autonomous agents may react in different ways to the same inputs over time, because, for instance they have changeable goals and knowledge. For this reason, we argue that testing of autonomous agents requires a procedure that caters for a wide range of test case contexts, and that can search for the most demanding of these test cases, even when they are not apparent to the agents’ developers. In this paper, we address this problem, introducing and evaluating an approach to testing autonomous agents that uses evolutionary optimization to generate demanding test cases.

Predicting structural models for silicon clusters

This article introduces an efficient method to generate structural models for medium-sized silicon clusters. Geometrical information obtained from previous investigations of small clusters is initially sorted and then introduced into our predictor algorithm in order to generate structural models for large clusters. The method predicts geometries whose binding energies are close (95%) to the corresponding value for the ground-state with very low computational cost. These predictions can be used as a very good initial guess for any global optimization algorithm. As a test case, information from clusters up to 14 atoms was used to predict good models for silicon clusters up to 20 atoms. We believe that the new algorithm may enhance the performance of most optimization methods whenever some previous information is available. (C) 2003 Wiley Periodicals, Inc.

Adaptive wireless multimedia communication systems

In recent years, due to the rapid convergence of multimedia services, Internet and wireless communications, there has been a growing trend of heterogeneity (in terms of channel bandwidths, mobility levels of terminals, end-user quality-of-service (QoS) requirements) for emerging integrated wired/wireless networks. Moreover, in nowadays systems, a multitude of users coexists within the same network, each of them with his own QoS requirement and bandwidth availability. In this framework, embedded source coding allowing partial decoding at various resolution is an appealing technique for multimedia transmissions. This dissertation includes my PhD research, mainly devoted to the study of embedded multimedia bitstreams in heterogenous networks, developed at the University of Bologna, advised by Prof. O. Andrisano and Prof. A. Conti, and at the University of California, San Diego (UCSD), where I spent eighteen months as a visiting scholar, advised by Prof. L. B. Milstein and Prof. P. C. Cosman. In order to improve the multimedia transmission quality over wireless channels, joint source and channel coding optimization is investigated in a 2D time-frequency resource block for an OFDM system. We show that knowing the order of diversity in time and/or frequency domain can assist image (video) coding in selecting optimal channel code rates (source and channel code rates). Then, adaptive modulation techniques, aimed at maximizing the spectral efficiency, are investigated as another possible solution for improving multimedia transmissions. For both slow and fast adaptive modulations, the effects of imperfect channel estimation errors are evaluated, showing that the fast technique, optimal in ideal systems, might be outperformed by the slow adaptive modulation, when a real test case is considered. Finally, the effects of co-channel interference and approximated bit error probability (BEP) are evaluated in adaptive modulation techniques, providing new decision regions concepts, and showing how the widely used BEP approximations lead to a substantial loss in the overall performance.

Entwicklung und Test eines Röntgeninterferometers auf der Basis von Übergangsstrahlung

In der vorliegenden Arbeit werden Entwicklung und Test einesneuartigen Interferometers mit zwei örtlich separierten,phasenkorrelierten Röntgenquellen zur Messung des Realteilsdes komplexen Brechungsindex von dünnen, freitragendenFolien beschrieben. Die Röntgenquellen sind zwei Folien, indenen relativistische Elektronen der Energie 855 MeVÜbergangsstrahlung erzeugen. Das am Mainzer Mikrotron MAMIrealisierte Interferometer besteht aus einer Berylliumfolieeiner Dicke von 10 Mikrometer und einer Nickel-Probefolieeiner Dicke von 2.1 Mikrometer. Die räumlichenInterferenzstrukturen werden als Funktion desFolienabstandes in einer ortsauflösenden pn-CCD nach derFourier-Analyse des Strahlungsimpulses mittels einesSilizium-Einkristallspektrometers gemessen. Die Phase derIntensitätsoszillationen enthält Informationen über dieDispersion, die die in der strahlaufwärtigen Folie erzeugteWelle in der strahlabwärtigen Probefolie erfährt. AlsFallstudie wurde die Dispersion von Nickel im Bereich um dieK-Absorptionskane bei 8333 eV, sowie bei Photonenenergien um9930 eV gemessen. Bei beiden Energien wurden deutlicheInterferenzstrukturen nachgewiesen, wobei die Kohärenz wegenWinkelmischungen mit steigendem Folienabstand bzw.Beobachtungswinkel abnimmt. Es wurden Anpassungen vonSimulationsrechnungen an die Messdaten durchgeführt, die diekohärenzvermindernden Effekte berücksichtigen. Aus diesenAnpassungen konnte bei beiden untersuchten Energien dieDispersion der Nickelprobe mit einer relativen Genauigkeitvon kleiner gleich 1.5 % in guter Übereinstimmung mit derLiteratur bestimmt werden.