Integrating software testing and run-time checking in an assertion verification framework

We have designed and implemented a framework that unifies unit testing and run-time verification (as well as static verification and static debugging). A key contribution of our approach is that a unified assertion language is used for all of these tasks. We first propose methods for compiling runtime checks for (parts of) assertions which cannot be verified at compile-time via program transformation. This transformation allows checking preconditions and postconditions, including conditional postconditions, properties at arbitrary program points, and certain computational properties. The implemented transformation includes several optimizations to reduce run-time overhead. We also propose a minimal addition to the assertion language which allows defining unit tests to be run in order to detect possible violations of the (partial) specifications expressed by the assertions. This language can express for example the input data for performing the unit tests or the number of times that the unit tests should be repeated. We have implemented the framework within the Ciao/CiaoPP system and effectively applied it to the verification of ISO-prolog compliance and to the detection of different types of bugs in the Ciao system source code. Several experimental results are presented that ¡Ilústrate different trade-offs among program size, running time, or levéis of verbosity of the messages shown to the user.

Context-sensitive multivariant assertion checking in modular programs

We propose a modular, assertion-based system for verification and debugging of large logic programs, together with several interesting models for checking assertions statically in modular programs, each with different characteristics and representing different trade-offs. Our proposal is a modular and multivariant extensión of our previously proposed abstract assertion checking model and we also report on its implementation in the CiaoPP system. In our approach, the specification of the program, given by a set of assertions, may be partial, instead of the complete specification required by raditional verification systems. Also, the system can deal with properties which cannot always be determined at compile-time. As a result, the proposed system needs to work with safe approximations: all assertions proved correct are guaranteed to be valid and all errors actual errors. The use of modular, context-sensitive static analyzers also allows us to introduce a new distinction between assertions checked in a particular context or checked in general.

Interval-based resource usage verification: Formalization and prototype

In an increasing number of applications (e.g., in embedded, real-time, or mobile systems) it is important or even essential to ensure conformance with respect to a specification expressing resource usages, such as execution time, memory, energy, or user-defined resources. In previous work we have presented a novel framework for data size-aware, static resource usage verification. Specifications can include both lower and upper bound resource usage functions. In order to statically check such specifications, both upper- and lower-bound resource usage functions (on input data sizes) approximating the actual resource usage of the program which are automatically inferred and compared against the specification. The outcome of the static checking of assertions can express intervals for the input data sizes such that a given specification can be proved for some intervals but disproved for others. After an overview of the approach in this paper we provide a number of novel contributions: we present a full formalization, and we report on and provide results from an implementation within the Ciao/CiaoPP framework (which provides a general, unified platform for static and run-time verification, as well as unit testing). We also generalize the checking of assertions to allow preconditions expressing intervals within which the input data size of a program is supposed to lie (i.e., intervals for which each assertion is applicable), and we extend the class of resource usage functions that can be checked.

An approach to profiling for run-time checking of computational properties and performance debugging in logic programs.

Although several profiling techniques for identifying performance bottlenecks in logic programs have been developed, they are generally not automatic and in most cases they do not provide enough information for identifying the root causes of such bottlenecks. This complicates using their results for guiding performance improvement. We present a profiling method and tool that provides such explanations. Our profiler associates cost centers to certain program elements and can measure different types of resource-related properties that affect performance, preserving the precedence of cost centers in the call graph. It includes an automatic method for detecting procedures that are performance bottlenecks. The profiling tool has been integrated in a previously developed run-time checking framework to allow verification of certain properties when they cannot be verified statically. The approach allows checking global computational properties which require complex instrumentation tracking information about previous execution states, such as, e.g., that the execution time accumulated by a given procedure is not greater than a given bound. We have built a prototype implementation, integrated it in the Ciao/CiaoPP system and successfully applied it to performance improvement, automatic optimization (e.g., resource-aware specialization of programs), run-time checking, and debugging of global computational properties (e.g., resource usage) in Prolog programs.

Integrating software testing and run-time checking in an assertion verification framework.

We have designed and implemented a framework that unifies unit testing and run-time verification (as well as static verification and static debugging). A key contribution of our approach is that a unified assertion language is used for all of these tasks. We first propose methods for compiling runtime checks for (parts of) assertions which cannot be verified at compile-time via program transformation. This transformation allows checking preconditions and postconditions, including conditional postconditions, properties at arbitrary program points, and certain computational properties. The implemented transformation includes several optimizations to reduce run-time overhead. We also propose a minimal addition to the assertion language which allows defining unit tests to be run in order to detect possible violations of the (partial) specifications expressed by the assertions. This language can express for example the input data for performing the unit tests or the number of times that the unit tests should be repeated. We have implemented the framework within the Ciao/CiaoPP system and effectively applied it to the verification of ISO-prolog compliance and to the detection of different types of bugs in the Ciao system source code. Several experimental results are presented that illustrate different trade-offs among program size, running time, or levels of verbosity of the messages shown to the user.

A Keyword-driven approach for generating OWL DL conformance test data

The conformance of semantic technologies has to be systematically evaluated to measure and verify the real adherence of these technologies to the Semantic Web standards. Currente valuations of semantic technology conformance are not exhaustive enough and do not directly cover user requirements and use scenarios, which raises the need for a simple, extensible and parameterizable method to generate test data for such evaluations. To address this need, this paper presents a keyword-driven approach for generating ontology language conformance test data that can be used to evaluate semantic technologies, details the definition of a test suite for evaluating OWL DL conformance using this approach,and describes the use and extension of this test suite during the evaluation of some tools.

Conformance monitoring, contract of objectives, tube in the sky

Ponencia invitada sobre asignacion y gestion de losts en el curso de verano de la UPM Research in Decisión Support Systems for future Air Traffic Management

Checking existing formulas for estimating overtopping rates in harbour breakwaters

The purpose of the research work resulting from various studies undertaken in the CEDEX, as summarized in this article, is to make a comparative analysis of methods for calculating overtopping rates developed by different authors. To this effect, in the first place, existing formulas for estimating the overtopping rate on rubble mound and vertical breakwaters were summarised and analysed. Later, the above mentioned formulas were compared using the results obtained in a series of hydraulic model tests at the CEDEX. The results obtained in the Ferrol outer harbour breakwater and Melilla harbour breakwater tests are presented here. A calculation method based on the neural network theory, developed in the European CLASH Project, was applied to a series of sloping breakwater tests in order to complete this research and the results obtained in the Ferrol outer harbour breakwater test are presented in this article. A series of additional tests was also carried out in a physical model on the standard cross section of the Bilbao harbour sloping breakwater’s cross section, the results of which are under study using the empirical formulas applicable to the cross section, as well as the NN-OVERTOPPING neural network

Checking the concurrence among macrobenthic organism distribution patterns at different taxonomic scales in relation to environmental factors

The interaction of both natural conditions and anthropogenic environmental impacts can lead to different soft-bottom macrobenthic distribution patterns. Soft-bottom macrobenthic community was analysed at different taxonomic scales in order to evaluate whether diverse subset of organisms respond to the variability of the environmental pressures (natural and human induced) showing or not similar distribution patterns. Therefore, this long-term survey had been focused on a heterogeneous area, where both anthropogenic and natural stress may affect the community. Three perpendicular transects to the coast were established and stations at 4, 10 and 15 m depths were sampled at each transect twice a year (summer- winter) from 2004 to 2009. Non-parametric multivariate techniques were used to analyse soft-bottom macrobenthic community distribution and its relation to the environmental factors. Similar distribution patterns between investigated taxonomic levels were detected and they were mainly related to depth.

Assessing the Implementation of a Secondary Plan Using a Conformance-Based GIS Method: A Pilot Project in Winnipeg, Manitoba

This case study evaluates the implementation of a secondary land use plan in Winnipeg, MB. The area selected for this case study is the Northeast Neighbourhood located in Waverley West; the development of this neighbourhood was guided by the Northeast Neighbourhood Area Structure Plan (NNASP). This case study evaluates the implementation of the NNASP through a conformance analysis which answers the following research questions: 1) Does the developed land use pattern in the NNASP area conform to what was planned; and 2) Does the implementation of the NNASP conform to the goals, objectives, policies, and intent of the plan? The implementation of the NNASP was evaluated against 62 evaluation criteria which were generated based on the policies of the NNASP. Using this method, the development of the Northeast Neighbourhood is effectively evaluated against the requirements of the NNASP. This conformity test utilized threefold approach including GIS analysis, a site visit, and document analysis.