Estudio preliminar acerca del uso de protocolos y actos comunicativos FIPA en el sistema COMPUTAPLEX

Este trabajo corresponde con la implementación de componentes software dentro de la Plataforma COMPUTAPLEX, la cual tiene como objetivo facilitar a los investigadores la realización de tareas del proceso experimental de ingeniería de software. Uno de los aportes a esta plataforma tecnológica corresponde con el desarrolló de los componentes necesarios para la recuperación de datos experimentales disponibles en diversas fuentes de datos, para ello se hizo uso de un mecanismo capaz de unificar la extracción de información de MySQL, ficheros excel y ficheros SPSS. Con ello diferentes grupos de investigación asociados pueden compartir y tener acceso a repositorios experimentales que se mantienen tanto de manera local como externa. Por otra parte, se ha realizado un estudio de la tecnología de agentes en la que se describe sus definiciones, lenguajes de comunicación, especificación FIPA, JADE como implementación FIPA y parser XML. Además para este trabajo se ha definido e implementado una ontología de comunicación entre agentes, la misma que fue diseñada en la herramienta Protégé. En lo que se refiere al desarrollo de componentes se hizo uso de una amplía variedad de tecnologías que incluye lenguaje de programación Java, framework JADE para el desarrollo de agentes, librería JENA para manejo de ontologías, librería SAXParser para lectura de archivos XML y patrón de diseño Factory. Finalmente se describe la metodología de trabajo utilizada en el proyecto, la cual por medio de la realización de varios ciclos iterativos permitió obtener prototipos que poco a poco fueron cubriendo las necesidades del producto software.----ABSTRACT---- This work relates to the implementation of software components within the platform Computaplex, which aims to enable researchers to conduct experimental software engineering process tasks. One of the contributions to this platform technology corresponds to the development of components which are necessary for the recovery of experimental data available in different data sources, to archive this goal a mechanism able to unify the extraction of information from MySQL, Excel and SPSS files was made. Therefore, associated research groups can share and access experimental repositories that remain both locally and externally. Moreover, it has been conducted a study of agent technology in its definition is described, languages communication, FIPA, JADE and FIPA implementation and XML parser. In addition to this work, it has been defined and implemented an ontology for communication between agents, the same as was designed in the Protégé tool. In what refers to the development of components, a wide range of technologies have been made which includes Java programming language, framework JADE for agent development, JENA library for handling ontologies, SAXParser for reading XML files and Factory design pattern. Finally, describing the work methodology used in this project, which through the implementation of several iterative cycles allowed to obtain prototypes were gradually meeting the needs of the software product.

Effectiveness for detecting faults within and outside the scope of testing techniques: an independent replication

The verification and validation activity plays a fundamental role in improving software quality. Determining which the most effective techniques for carrying out this activity are has been an aspiration of experimental software engineering researchers for years. This paper reports a controlled experiment evaluating the effectiveness of two unit testing techniques (the functional testing technique known as equivalence partitioning (EP) and the control-flow structural testing technique known as branch testing (BT)). This experiment is a literal replication of Juristo et al. (2013). Both experiments serve the purpose of determining whether the effectiveness of BT and EP varies depending on whether or not the faults are visible for the technique (InScope or OutScope, respectively). We have used the materials, design and procedures of the original experiment, but in order to adapt the experiment to the context we have: (1) reduced the number of studied techniques from 3 to 2; (2) assigned subjects to experimental groups by means of stratified randomization to balance the influence of programming experience; (3) localized the experimental materials and (4) adapted the training duration. We ran the replication at the Escuela Polite?cnica del Eje?rcito Sede Latacunga (ESPEL) as part of a software verification & validation course. The experimental subjects were 23 master?s degree students. EP is more effective than BT at detecting InScope faults. The session/program and group variables are found to have significant effects. BT is more effective than EP at detecting OutScope faults. The session/program and group variables have no effect in this case. The results of the replication and the original experiment are similar with respect to testing techniques. There are some inconsistencies with respect to the group factor. They can be explained by small sample effects. The results for the session/program factor are inconsistent for InScope faults. We believe that these differences are due to a combination of the fatigue effect and a technique x program interaction. Although we were able to reproduce the main effects, the changes to the design of the original experiment make it impossible to identify the causes of the discrepancies for sure. We believe that further replications closely resembling the original experiment should be conducted to improve our understanding of the phenomena under study.

Identification and correction of besouro's faults: impact on test‐driven development experiments

Context. This thesis is framed in experimental software engineering. More concretely, it addresses the problems arisen when assessing process conformance in test-driven development experiments conducted by UPM's Experimental Software Engineering group. Process conformance was studied using the Eclipse's plug-in tool Besouro. It has been observed that Besouro does not work correctly in some circumstances. It creates doubts about the correction of the existing experimental data which render it useless. Aim. The main objective of this work is the identification and correction of Besouro's faults. A secondary goal is fixing the datasets already obtained in past experiments to the maximum possible extent. This way, existing experimental results could be used with confidence. Method. (1) Testing Besouro using different sequences of events (creation methods, assertions etc..) to identify the underlying faults. (2) Fix the code and (3) fix the datasets using code specially created for this purpose. Results. (1) We confirmed the existence of several fault in Besouro's code that affected to Test-First and Test-Last episode identification. These faults caused the incorrect identification of 20% of episodes. (2) We were able to fix Besouro's code. (3) The correction of existing datasets was possible, subjected to some restrictions (such us the impossibility of tracing code size increase to programming time. Conclusion. The results of past experiments dependent upon Besouro's data could no be trustable. We have the suspicion that more faults remain in Besouro's code, whose identification requires further analysis.

Gestión de configuración y línea de productos para mejorar el proceso experimental en ingeniería del software

La Ingeniería del Software (IS) Empírica adopta el método científico a la IS para facilitar la generación de conocimiento. Una de las técnicas empleadas, es la realización de experimentos. Para que el conocimiento obtenido experimentalmente adquiera el nivel de madurez necesario para su posterior uso, es necesario que los experimentos sean replicados. La existencia de múltiples replicaciones de un mismo experimento conlleva la existencia de numerosas versiones de los distintos productos generados durante la realización de cada replicación. Actualmente existe un gran descontrol sobre estos productos, ya que la administración se realiza de manera informal. Esto causa problemas a la hora de planificar nuevas replicaciones, o intentar obtener información sobre las replicaciones ya realizadas. Para conocer con detalle la dimensión del problema a resolver, se estudia el estado actual de la gestión de materiales experimentales y su uso en replicaciones, así como de las herramientas de gestión de materiales experimentales. El estudio concluye que ninguno de los enfoques estudiados proporciona una solución al problema planteado. Este trabajo persigue como objetivo mejorar la administración de los materiales experimentales y replicaciones de experimentos en IS para dar soporte a la replicación de experimentos. Para satisfacer este objetivo, se propone la adopción en experimentación de los paradigmas de Gestión de Configuración del Software (GCS) y Línea de Producto Software (LPS). Para desarrollar la propuesta se decide utilizar el método de investigación acción (en inglés action research). Para adoptar la GCS a experimentación, se comienza realizando un estudio del proceso experimental como transformación de productos; a continuación, se realiza una adopción de conceptos fundamentada en los procesos del desarrollo software y de experimentación; finalmente, se desarrollan un conjunto de instrumentos, que se incorporan a un Plan de Gestión de Configuración de Experimentos (PGCE). Para adoptar la LPS a experimentación, se comienza realizando un estudio de los conceptos, actividades y fases que fundamentan la LPS; a continuación, se realiza una adopción de los conceptos; finalmente, se desarrollan o adoptan las técnicas, simbología y modelos para dar soporte a las fases de la Línea de Producto para Experimentación (LPE). La propuesta se valida mediante la evaluación de su: viabilidad, flexibilidad, usabilidad y satisfacción. La viabilidad y flexibilidad se evalúan mediante la instanciación del PGCE y de la LPE en experimentos concretos en IS. La usabilidad se evalúa mediante el uso de la propuesta para la generación de las instancias del PGCE y de LPE. La satisfacción evalúa la información sobre el experimento que contiene el PGCE y la LPE. Los resultados de la validación de la propuesta muestran mejores resultados en los aspectos de usabilidad y satisfacción a los experimentadores. ABSTRACT Empirical software engineering adapts the scientific method to software engineering (SE) in order to facilitate knowledge generation. Experimentation is one of the techniques used. For the knowledge generated experimentally to acquire the level of maturity necessary for later use, the experiments have to be replicated. As the same experiment is replicated more than once, there are numerous versions of all the products generated during a replication. These products are generally administered informally without control. This is troublesome when it comes to planning new replications or trying to gather information on replications conducted in the past. In order to grasp the size of the problem to be solved, this research examines the current state of the art of the management and use of experimental materials in replications, as well as the tools managing experimental materials. The study concludes that none of the analysed approaches provides a solution to the stated problem. The aim of this research is to improve the administration of SE experimental materials and experimental replications in support of experiment replication. To do this, we propose the adaptation of software configuration management (SCM) and software product line (SPL) paradigms to experimentation. The action research method was selected in order to develop this proposal. The first step in the adaptation of the SCM to experimentation was to analyse the experimental process from the viewpoint of the transformation of products. The concepts were then adapted based on software development and experimentation processes. Finally, a set of instruments were developed and added to an experiment configuration management plan (ECMP). The first step in the adaptation of the SPL to experimentation is to analyse the concepts, activities and phases underlying the SPL. The concepts are then adapted. Finally, techniques, symbols and models are developed or adapted in support of the experimentation product line (EPL) phases. The proposal is validated by evaluating its feasibility, flexibility, usability and satisfaction. Feasibility and flexibility are evaluated by instantiating the ECMP and the EPL in specific SE experiments. Usability is evaluated by using the proposal to generate the instances of the ECMP and EPL. The results of the validation of the proposal show that the proposal performs better with respect to usability issues and experimenter satisfaction.

Conceptualización e Infraestructura para la Investigación Experimental en Ingeniería del Software

Antecedentes: Esta investigación se enmarca principalmente en la replicación y secundariamente en la síntesis de experimentos en Ingeniería de Software (IS). Para poder replicar, es necesario disponer de todos los detalles del experimento original. Sin embargo, la descripción de los experimentos es habitualmente incompleta debido a la existencia de conocimiento tácito y a la existencia de otros problemas tales como: La carencia de un formato estándar de reporte, la inexistencia de herramientas que den soporte a la generación de reportes experimentales, etc. Esto provoca que no se pueda reproducir fielmente el experimento original. Esta problemática limita considerablemente la capacidad de los experimentadores para llevar a cabo replicaciones y por ende síntesis de experimentos. Objetivo: La investigación tiene como objetivo formalizar el proceso experimental en IS, de modo que facilite la comunicación de información entre experimentadores. Contexto: El presente trabajo de tesis doctoral ha sido desarrollado en el seno del Grupo de Investigación en Ingeniería del Software Empírica (GrISE) perteneciente a la Escuela Técnica Superior de Ingenieros Informáticos (ETSIINF) de la Universidad Politécnica de Madrid (UPM), como parte del proyecto TIN2011-23216 denominado “Tecnologías para la Replicación y Síntesis de Experimentos en Ingeniería de Software”, el cual es financiado por el Gobierno de España. El grupo GrISE cumple a la perfección con los requisitos necesarios (familia de experimentos establecida, con al menos tres líneas experimentales y una amplia experiencia en replicaciones (16 replicaciones hasta 2011 en la línea de técnicas de pruebas de software)) y ofrece las condiciones para que la investigación se lleve a cabo de la mejor manera, como por ejemplo, el acceso total a su información. Método de Investigación: Para cumplir este objetivo se opta por Action Research (AR) como el método de investigación más adecuado a las características de la investigación, para obtener resultados a través de aproximaciones sucesivas que abordan los problemas concretos de comunicación entre experimentadores. Resultados: Se formalizó el modelo conceptual del ciclo experimental desde la perspectiva de los 3 roles principales que representan los experimentadores en el proceso experimental, siendo estos: Gestor de la Investigación (GI), Gestor del Experimento (GE) y Experimentador Senior (ES). Por otra parte, se formalizó el modelo del ciclo experimental, a través de: Un workflow del ciclo y un diagrama de procesos. Paralelamente a la formalización del proceso experimental en IS, se desarrolló ISRE (de las siglas en inglés Infrastructure for Sharing and Replicating Experiments), una prueba de concepto de entorno de soporte a la experimentación en IS. Finalmente, se plantearon guías para el desarrollo de entornos de soporte a la experimentación en IS, en base al estudio de las características principales y comunes de los modelos de las herramientas de soporte a la experimentación en distintas disciplinas experimentales. Conclusiones: La principal contribución de la investigación esta representada por la formalización del proceso experimental en IS. Los modelos que representan la formalización del ciclo experimental, así como la herramienta ISRE, construida a modo de evaluación de los modelos, fueron encontrados satisfactorios por los experimentadores del GrISE. Para consolidar la validez de la formalización, consideramos que este estudio debería ser replicado en otros grupos de investigación representativos en la comunidad de la IS experimental. Futuras Líneas de Investigación: El cumplimiento de los objetivos, de la mano con los hallazgos alcanzados, han dado paso a nuevas líneas de investigación, las cuales son las siguientes: (1) Considerar la construcción de un mecanismo para facilitar el proceso de hacer explícito el conocimiento tácito de los experimentadores por si mismos de forma colaborativa y basados en el debate y el consenso , (2) Continuar la investigación empírica en el mismo grupo de investigación hasta cubrir completamente el ciclo experimental (por ejemplo: experimentos nuevos, síntesis de resultados, etc.), (3) Replicar el proceso de investigación en otros grupos de investigación en ISE, y (4) Renovar la tecnología de la prueba de concepto, tal que responda a las restricciones y necesidades de un entorno real de investigación. ABSTRACT Background: This research addresses first and foremost the replication and also the synthesis of software engineering (SE) experiments. Replication is impossible without access to all the details of the original experiment. But the description of experiments is usually incomplete because knowledge is tacit, there is no standard reporting format or there are hardly any tools to support the generation of experimental reports, etc. This means that the original experiment cannot be reproduced exactly. These issues place considerable constraints on experimenters’ options for carrying out replications and ultimately synthesizing experiments. Aim: The aim of the research is to formalize the SE experimental process in order to facilitate information communication among experimenters. Context: This PhD research was developed within the empirical software engineering research group (GrISE) at the Universidad Politécnica de Madrid (UPM)’s School of Computer Engineering (ETSIINF) as part of project TIN2011-23216 entitled “Technologies for Software Engineering Experiment Replication and Synthesis”, which was funded by the Spanish Government. The GrISE research group fulfils all the requirements (established family of experiments with at least three experimental lines and lengthy replication experience (16 replications prior to 2011 in the software testing techniques line)) and provides favourable conditions for the research to be conducted in the best possible way, like, for example, full access to information. Research Method: We opted for action research (AR) as the research method best suited to the characteristics of the investigation. Results were generated successive rounds of AR addressing specific communication problems among experimenters. Results: The conceptual model of the experimental cycle was formalized from the viewpoint of three key roles representing experimenters in the experimental process. They were: research manager, experiment manager and senior experimenter. The model of the experimental cycle was formalized by means of a workflow and a process diagram. In tandem with the formalization of the SE experimental process, infrastructure for sharing and replicating experiments (ISRE) was developed. ISRE is a proof of concept of a SE experimentation support environment. Finally, guidelines for developing SE experimentation support environments were designed based on the study of the key features that the models of experimentation support tools for different experimental disciplines had in common. Conclusions: The key contribution of this research is the formalization of the SE experimental process. GrISE experimenters were satisfied with both the models representing the formalization of the experimental cycle and the ISRE tool built in order to evaluate the models. In order to further validate the formalization, this study should be replicated at other research groups representative of the experimental SE community. Future Research Lines: The achievement of the aims and the resulting findings have led to new research lines, which are as follows: (1) assess the feasibility of building a mechanism to help experimenters collaboratively specify tacit knowledge based on debate and consensus, (2) continue empirical research at the same research group in order to cover the remainder of the experimental cycle (for example, new experiments, results synthesis, etc.), (3) replicate the research process at other ESE research groups, and (4) update the tools of the proof of concept in order to meet the constraints and needs of a real research environment.

The effects of normalisation on the ability of business end-users to detect data anomalies: An experimental evaluation

With the proliferation of relational database programs for PC's and other platforms, many business end-users are creating, maintaining, and querying their own databases. More importantly, business end-users use the output of these queries as the basis for operational, tactical, and strategic decisions. Inaccurate data reduce the expected quality of these decisions. Implementing various input validation controls, including higher levels of normalisation, can reduce the number of data anomalies entering the databases. Even in well-maintained databases, however, data anomalies will still accumulate. To improve the quality of data, databases can be queried periodically to locate and correct anomalies. This paper reports the results of two experiments that investigated the effects of different data structures on business end-users' abilities to detect data anomalies in a relational database. The results demonstrate that both unnormalised and higher levels of normalisation lower the effectiveness and efficiency of queries relative to the first normal form. First normal form databases appear to provide the most effective and efficient data structure for business end-users formulating queries to detect data anomalies.

Information technology service management: an experimental approach towards IT service prediction

Dissertation presented to obtain a Masters degree in Computer Science

Derivation and consistency checking of models in early software product line engineering

Dissertação para obtenção do Grau de Doutor em Engenharia Informática

Selected Aspects Related to Pre-development Stages of International Software projects. Case of Finland and Russia

The question of Pilot Project creation, due to support pre-development stage of software product elaboration, nowadays might be used as an approach, which allows improving the whole scheme of information technology project running. This subject is not new, but till now no model has been presented, which gives deep description of this important stage on the early phase of project. This Master's Thesis represents the research's results and findings concerning the pre-development study from the Software Engineering point of view. The aspects of feasibility study, pilot prototype developments are analyzed in this paper. As the result, the technique of Pilot Project is formulated and scheme has been presented. The experimental part is focused on particular area Pilot Project scheme's implementation- Internationally Distributed Software projects. The specific characteristic, aspects, obstacles, advantages and disadvantages are considered on the example of cross border region of Russia and Finland. The real case of Pilot Project technique implementation is given.

Including hardware/software co-design in the ASSERT model driven engineering process.

Abstract. The ASSERT project de?ned new software engineering methods and tools for the development of critical embedded real-time systems in the space domain. The ASSERT model-driven engineering process was one of the achievements of the project and is based on the concept of property- preserving model transformations. The key element of this process is that non-functional properties of the software system must be preserved during model transformations. Properties preservation is carried out through model transformations compliant with the Ravenscar Pro?le and provides a formal basis to the process. In this way, the so-called Ravenscar Computational Model is central to the whole ASSERT process. This paper describes the work done in the HWSWCO study, whose main objective has been to address the integration of the Hardware/Software co-design phase in the ASSERT process. In order to do that, non-functional properties of the software system must also be preserved during hardware synthesis. Keywords : Ada 2005, Ravenscar pro?le, Hardware/Software co-design, real- time systems, high-integrity systems, ORK

Reviewing technical approaches for sharing and preservation of experimental data

Empirical Software Engineering (ESE) replication researchers need to store and manipulate experimental data for several purposes, in particular analysis and reporting. Current research needs call for sharing and preservation of experimental data as well. In a previous work, we analyzed Replication Data Management (RDM) needs. A novel concept, called Experimental Ecosystem, was proposed to solve current deficiencies in RDMapproaches. The empirical ecosystem provides replication researchers with a common framework that integrates transparently local heterogeneous data sources. A typical situation where the Empirical Ecosystem is applicable, is when several members of a research group, or several research groups collaborating together, need to share and access each other experimental results. However, to be able to apply the Empirical Ecosystem concept and deliver all promised benefits, it is necessary to analyze the software architectures and tools that can properly support it.

Are team personality and climate related to satisfaction and software quality? Aggregating results from a twice replicated experiment

Research into software engineering teams focuses on human and social team factors. Social psychology deals with the study of team formation and has found that personality factors and group processes such as team climate are related to team effectiveness. However, there are only a handful of empirical studies dealing with personality and team climate and their relationship to software development team effectiveness. Objective We present aggregate results of a twice replicated quasi-experiment that evaluates the relationships between personality, team climate, product quality and satisfaction in software development teams. Method Our experimental study measures the personalities of team members based on the Big Five personality traits (openness, conscientiousness, extraversion, agreeableness, neuroticism) and team climate factors (participative safety, support for innovation, team vision and task orientation) preferences and perceptions. We aggregate the results of the three studies through a meta-analysis of correlations. The study was conducted with students. Results The aggregation of results from the baseline experiment and two replications corroborates the following findings. There is a positive relationship between all four climate factors and satisfaction in software development teams. Teams whose members score highest for the agreeableness personality factor have the highest satisfaction levels. The results unveil a significant positive correlation between the extraversion personality factor and software product quality. High participative safety and task orientation climate perceptions are significantly related to quality. Conclusions First, more efficient software development teams can be formed heeding personality factors like agreeableness and extraversion. Second, the team climate generated in software development teams should be monitored for team member satisfaction. Finally, aspects like people feeling safe giving their opinions or encouraging team members to work hard at their job can have an impact on software quality. Software project managers can take advantage of these factors to promote developer satisfaction and improve the resulting product.

Ex ante evaluations of alternate data structures for end user queries: Theory and experimental test

The data structure of an information system can significantly impact the ability of end users to efficiently and effectively retrieve the information they need. This research develops a methodology for evaluating, ex ante, the relative desirability of alternative data structures for end user queries. This research theorizes that the data structure that yields the lowest weighted average complexity for a representative sample of information requests is the most desirable data structure for end user queries. The theory was tested in an experiment that compared queries from two different relational database schemas. As theorized, end users querying the data structure associated with the less complex queries performed better Complexity was measured using three different Halstead metrics. Each of the three metrics provided excellent predictions of end user performance. This research supplies strong evidence that organizations can use complexity metrics to evaluate, ex ante, the desirability of alternate data structures. Organizations can use these evaluations to enhance the efficient and effective retrieval of information by creating data structures that minimize end user query complexity.

Maximising the information gained from an experimental analysis of code inspection and static analysis for concurrent Java components

The results of empirical studies are limited to particular contexts, difficult to generalise and the studies themselves are expensive to perform. Despite these problems, empirical studies in software engineering can be made effective and they are important to both researchers and practitioners. The key to their effectiveness lies in the maximisation of the information that can be gained by examining existing studies, conducting power analyses for an accurate minimum sample size and benefiting from previous studies through replication. This approach was applied in a controlled experiment examining the combination of automated static analysis tools and code inspection in the context of verification and validation (V&V) of concurrent Java components. The combination of these V&V technologies was shown to be cost-effective despite the size of the study, which thus contributes to research in V&V technology evaluation.