Model-based testing of software systems : functionality and performance

Software is a key component in many of our devices and products that we use every day. Most customers demand not only that their devices should function as expected but also that the software should be of high quality, reliable, fault tolerant, efficient, etc. In short, it is not enough that a calculator gives the correct result of a calculation, we want the result instantly, in the right form, with minimal use of battery, etc. One of the key aspects for succeeding in today's industry is delivering high quality. In most software development projects, high-quality software is achieved by rigorous testing and good quality assurance practices. However, today, customers are asking for these high quality software products at an ever-increasing pace. This leaves the companies with less time for development. Software testing is an expensive activity, because it requires much manual work. Testing, debugging, and verification are estimated to consume 50 to 75 per cent of the total development cost of complex software projects. Further, the most expensive software defects are those which have to be fixed after the product is released. One of the main challenges in software development is reducing the associated cost and time of software testing without sacrificing the quality of the developed software. It is often not enough to only demonstrate that a piece of software is functioning correctly. Usually, many other aspects of the software, such as performance, security, scalability, usability, etc., need also to be verified. Testing these aspects of the software is traditionally referred to as nonfunctional testing. One of the major challenges with non-functional testing is that it is usually carried out at the end of the software development process when most of the functionality is implemented. This is due to the fact that non-functional aspects, such as performance or security, apply to the software as a whole. In this thesis, we study the use of model-based testing. We present approaches to automatically generate tests from behavioral models for solving some of these challenges. We show that model-based testing is not only applicable to functional testing but also to non-functional testing. In its simplest form, performance testing is performed by executing multiple test sequences at once while observing the software in terms of responsiveness and stability, rather than the output. The main contribution of the thesis is a coherent model-based testing approach for testing functional and performance related issues in software systems. We show how we go from system models, expressed in the Unified Modeling Language, to test cases and back to models again. The system requirements are traced throughout the entire testing process. Requirements traceability facilitates finding faults in the design and implementation of the software. In the research field of model-based testing, many new proposed approaches suffer from poor or the lack of tool support. Therefore, the second contribution of this thesis is proper tool support for the proposed approach that is integrated with leading industry tools. We o er independent tools, tools that are integrated with other industry leading tools, and complete tool-chains when necessary. Many model-based testing approaches proposed by the research community suffer from poor empirical validation in an industrial context. In order to demonstrate the applicability of our proposed approach, we apply our research to several systems, including industrial ones.

Development of Measurement Systems in Scientific Research: Case Study

In recent years, technological advancements in microelectronics and sensor technologies have revolutionized the field of electrical engineering. New manufacturing techniques have enabled a higher level of integration that has combined sensors and electronics into compact and inexpensive systems. Previously, the challenge in measurements was to understand the operation of the electronics and sensors, but this has now changed. Nowadays, the challenge in measurement instrumentation lies in mastering the whole system, not just the electronics. To address this issue, this doctoral dissertation studies whether it would be beneficial to consider a measurement system as a whole from the physical phenomena to the digital recording device, where each piece of the measurement system affects the system performance, rather than as a system consisting of small independent parts such as a sensor or an amplifier that could be designed separately. The objective of this doctoral dissertation is to describe in depth the development of the measurement system taking into account the challenges caused by the electrical and mechanical requirements and the measurement environment. The work is done as an empirical case study in two example applications that are both intended for scientific studies. The cases are a light sensitive biological sensor used in imaging and a gas electron multiplier detector for particle physics. The study showed that in these two cases there were a number of different parts of the measurement system that interacted with each other. Without considering these interactions, the reliability of the measurement may be compromised, which may lead to wrong conclusions about the measurement. For this reason it is beneficial to conceptualize the measurement system as a whole from the physical phenomena to the digital recording device where each piece of the measurement system affects the system performance. The results work as examples of how a measurement system can be successfully constructed to support a study of sensors and electronics.

Formal development of resilient distributed systems

Resilience is the property of a system to remain trustworthy despite changes. Changes of a different nature, whether due to failures of system components or varying operational conditions, significantly increase the complexity of system development. Therefore, advanced development technologies are required to build robust and flexible system architectures capable of adapting to such changes. Moreover, powerful quantitative techniques are needed to assess the impact of these changes on various system characteristics. Architectural flexibility is achieved by embedding into the system design the mechanisms for identifying changes and reacting on them. Hence a resilient system should have both advanced monitoring and error detection capabilities to recognise changes as well as sophisticated reconfiguration mechanisms to adapt to them. The aim of such reconfiguration is to ensure that the system stays operational, i.e., remains capable of achieving its goals. Design, verification and assessment of the system reconfiguration mechanisms is a challenging and error prone engineering task. In this thesis, we propose and validate a formal framework for development and assessment of resilient systems. Such a framework provides us with the means to specify and verify complex component interactions, model their cooperative behaviour in achieving system goals, and analyse the chosen reconfiguration strategies. Due to the variety of properties to be analysed, such a framework should have an integrated nature. To ensure the system functional correctness, it should rely on formal modelling and verification, while, to assess the impact of changes on such properties as performance and reliability, it should be combined with quantitative analysis. To ensure scalability of the proposed framework, we choose Event-B as the basis for reasoning about functional correctness. Event-B is a statebased formal approach that promotes the correct-by-construction development paradigm and formal verification by theorem proving. Event-B has a mature industrial-strength tool support { the Rodin platform. Proof-based verification as well as the reliance on abstraction and decomposition adopted in Event-B provides the designers with a powerful support for the development of complex systems. Moreover, the top-down system development by refinement allows the developers to explicitly express and verify critical system-level properties. Besides ensuring functional correctness, to achieve resilience we also need to analyse a number of non-functional characteristics, such as reliability and performance. Therefore, in this thesis we also demonstrate how formal development in Event-B can be combined with quantitative analysis. Namely, we experiment with integration of such techniques as probabilistic model checking in PRISM and discrete-event simulation in SimPy with formal development in Event-B. Such an integration allows us to assess how changes and di erent recon guration strategies a ect the overall system resilience. The approach proposed in this thesis is validated by a number of case studies from such areas as robotics, space, healthcare and cloud domain.

Product Assurance in Agile Space Systems Development

Product assurance is an essential part of product development process if developers want to ensure that final product is safe and reliable. Product assurance can be supported withrisk management and with different failure analysis methods. Product assurance is emphasized in system development process of mission critical systems. The product assurance process in systems of this kind requires extra attention. Inthis thesis, mission critical systems are space systems and the product assurance processof these systems is presented with help of space standards. The product assurance process can be supported with agile development because agile emphasizes transparency of the process and fast response to changes. Even if the development process of space systems is highly standardized and reminds waterfall model, it is still possible to adapt agile development in space systems development. This thesisaims to support the product assurance process of space systems with agile developmentso that the final product would be as safe and reliable as possible. The main purpose of this thesis is to examine how well product assurance is performed in Finnish space organizations and how product assurance tasks and activities can besupported with agile development. The research part of this thesis is performed in survey form.

Product assurance in agile space systems development

Product assurance is an essential part of product development process if developers want to ensure that final product is safe and reliable. Product assurance can be supported with risk management and with different failure analysis methods. Product assurance is emphasized in system development process of mission critical systems. The product assurance process in systems of this kind requires extra attention. In this thesis, mission critical systems are space systems and the product assurance process of these systems is presented with help of space standards. The product assurance process can be supported with agile development because agile emphasizes transparency of the process and fast response to changes. Even if the development process of space systems is highly standardized and reminds waterfall model, it is still possible to adapt agile development in space systems development. This thesis aims to support the product assurance process of space systems with agile development so that the final product would be as safe and reliable as possible. The main purpose of this thesis is to examine how well product assurance is performed in Finnish space organizations and how product assurance tasks and activities can be supported with agile development. The research part of this thesis is performed in survey form.

Choosing the service support model after a merger: Case Are Ltd.

The meaning of information technology (IT) and information systems have increased during the last few years. This is mainly because business is nowadays seen more and more as a service business and IT is one of the key elements to support those business services. Since the meaning of IT services has increased also the meaning of IT service support should be a factor paid more attention to. Especially after a merger and acquisition (M&A) it is more important than ever to consider service support. The purpose of this study is to discover the best practices for choosing a suitable service support model. The research question is How to choose a service support organization model for the ERP service desk function after a merger? A qualitative method is selected as a research method. This thesis includes two parts: a literature review and a case study. Theoretical part compiles an integrated model of previous research on the topic. It consists a collection of academic articles, publications and reports. The empirical part focuses on the issues in the case organization. That part tries to answer the question: what would be the most suitable service support model for the case organization? The empirical part is conducted by interviewing the employees of the case organization. This study finds that even though there are many ways of selecting a service support model it is difficult to define an unambiguous guidelines. However, there are few main objectives that should be taken into account regardless the case. Especially by using ITIL processes it is possible to implement a comprehensive service support and raise overall awareness of the existing service support models. The main functions that need to be taken into account are nature, industry and size of the organization. Also the business strategy, goals and resources need to be considered. These are the same factors that are noticed in the case study as well. The suggestions for the case organization are presented based on the interviews and the literature review.

The role of management accounting systems and venture capital investors for start-up firms

The role of star-up firms in economy and the importance of venture capital investors for the growth of start-up firms have been highlighted in recent years. The growth challenges of start-up firms consist of fast changing environment, the availability of venture capital funding and the development of firm management in the growth phase. A growing number of studies have focused on management accounting systems and venture capital in start-up and growth firm context. In this thesis the role of management accounting systems and venture capital investors is considered in the growth phase of firm development. The theoretical objective of this thesis is to construct a theoretical framework in order to describe the importance of management accounting systems and venture capital investors in start-up firms. The practice orientated objective of this thesis is to study the application of management accounting systems and management accounting based information in start-up firms in high-technology industry as well as the impact of venture capital for management accounting system design. In addition, the growth challenges of start-up firms are studied in order to understand the context in which management accounting systems are used. The research approach of theoretical part is conceptual as the theoretical framework is constructed by combining literature on firm growth, management accounting and venture finance in order to analyse the phenomenon. The action-orientated research approach is appropriate for analysing and describing of the studied phenomenon through empirical evidence. The empirical evidence was collected through interviews with three experts in start-up firm accelerator centers, four representatives of start-up firms and one venture capital investor. The results indicate that the growth challenges of stat-up firms are not related to the development of management accounting systems. Managers of start-up firms expressed a positive attitude towards management accounting systems that improve efficiency of operations. In start-up firms flexible and adjustable management accounting practices, such as budgets, cash flow calculations and future-orientated analysis tools, are applied that support planning and coordination of operations. The results indicate that venture capital investors affect the provision and the quality of management accounting information during the investment process. In addition, venture capital investors enhance the use of management accounting information for internal coordination in start-up firms. By applying the theoretical framework in the analysis, it can be stated that by acting as support function management accounting systems facilitate start-up firms development.

Key factors in U.S. Government support of business internationalization in Finland Case: U.S. Embassy in Helsinki

This study discusses the importance of government intervention for companies and the expansion of national innovation systems. The purpose of the study is to examine the impact which the U.S. Embassy in Helsinki can have on Finnish businesses through their business support programs and events. The embassy has shifted focus in recent years with the creation of an innovation center and increased business services. The study has sub-objectives to discover the critical factors for producing impact, host and home based factor interaction, and effects produced by these initiatives. The theoretical background of the study consists of literature relating to the concepts of national innovation systems and government intervention. The empirical research conduct for this study is based on interviews with experts from the environment surrounding the U.S. Embassy in Helsinki, Finland and participation in embassy events. The data was collected between March 2014 and September 2015. Seven interviews were conducted; five with representatives of the U.S. Embassy and two with related organizations. Thematic analysis was used to categorize and interpret interview and observation data. The use of an impact radar was implemented as a basis for analysis. This study finds that the internationalization of national innovation systems provides interesting opportunities and challenges for national governments. The opportunity to provide services to foreign companies by an embassy in a stable environment opens the possibility to create positive notice and relations with the host country. The increased connections and inputs to the national innovation system of the home country have the potential to increase knowledge absorption and create positive growth. The most effective way for governments to encourage businesses is to create incentives and reduce barriers. The services are best aimed at small to medium sized companies in the early stages of development. The findings of this report suggest that the most critical factors for producing impact on companies are the ability to disseminate information effectively, the ability to create a positive image of the country, the ability to foster effective networks between the two countries, and the ability to facilitate the internationalization of companies. In the best cases, the embassy is able to create incentives to internationalize to the United States and reduce barriers which are encountered by companies. Future research is necessary to fully understand the impact of business services provided by an embassy can have on the political and economic relations of countries, and on particular industry sectors. The institutional setting provided by the embassy’s focus on business relations provides a rich environment for further study in a number of areas.