Introducing Continuous Integration for C and C++ Software Development Projects on Linux Platform

Software integration is a stage in a software development process to assemble separate components to produce a single product. It is important to manage the risks involved and being able to integrate smoothly, because software cannot be released without integrating it first. Furthermore, it has been shown that the integration and testing phase can make up 40 % of the overall project costs. These issues can be mitigated by using a software engineering practice called continuous integration. This thesis work presents how continuous integration is introduced to the author's employer organisation. This includes studying how the continuous integration process works and creating the technical basis to start using the process on future projects. The implemented system supports software written in C and C++ programming languages on Linux platform, but the general concepts can be applied to any programming language and platform by selecting the appropriate tools. The results demonstrate in detail what issues need to be solved when the process is acquired in a corporate environment. Additionally, they provide an implementation and process description suitable to the organisation. The results show that continuous integration can reduce the risks involved in a software process and increase the quality of the product as well.

Pilvisovelluskehitys Google App Engine -pilvisovellusalustalla

Tämä Diplomityö keskittyy tutkimaan pilvisovelluskehitystä Google App Engine – pilvisovellusalustalle perustuen kuusi –vaiheiseen vesiputousmalliin sekä tutkimaan Google App Engine -pilvisovellusalustan tarjoamia mahdollisuuksia ja rajoituksia sovelluskehityksen muodossa. Tutkimuksen perusteella kuusi –vaiheinen vesiputousmalli soveltuu pilvisovelluskehitykseen,mikäli vaatimusmäärittely on tarkka jo sovelluskehityksen alkuvaiheessa. Tutkimuksen tuloksena syntyi vaatimusmäärittely MikkoMail –pilvisovellukselle. Vaatimusmäärittelyn pohjalta luotiin MikkoMail –pilvisovellus Google App Engine –pilvisovellusalustalle. Google App Engine –pilvisovellusalusta tukee vain Python- ja Java –ohjelmointikieliä eikä sisällä lainkaan ulkoista tietokantapalveluiden tukea. Tästä syystä Google App Engine -pilvisovellusalusta soveltuu pieniin, keskisuuriin ja pilottiprojektinomaisiin sovelluskehitysprojekteihin.

Scheduling dynamic dataflow graphs with model checking

With the shift towards many-core computer architectures, dataflow programming has been proposed as one potential solution for producing software that scales to a varying number of processor cores. Programming for parallel architectures is considered difficult as the current popular programming languages are inherently sequential and introducing parallelism is typically up to the programmer. Dataflow, however, is inherently parallel, describing an application as a directed graph, where nodes represent calculations and edges represent a data dependency in form of a queue. These queues are the only allowed communication between the nodes, making the dependencies between the nodes explicit and thereby also the parallelism. Once a node have the su cient inputs available, the node can, independently of any other node, perform calculations, consume inputs, and produce outputs. Data ow models have existed for several decades and have become popular for describing signal processing applications as the graph representation is a very natural representation within this eld. Digital lters are typically described with boxes and arrows also in textbooks. Data ow is also becoming more interesting in other domains, and in principle, any application working on an information stream ts the dataflow paradigm. Such applications are, among others, network protocols, cryptography, and multimedia applications. As an example, the MPEG group standardized a dataflow language called RVC-CAL to be use within reconfigurable video coding. Describing a video coder as a data ow network instead of with conventional programming languages, makes the coder more readable as it describes how the video dataflows through the different coding tools. While dataflow provides an intuitive representation for many applications, it also introduces some new problems that need to be solved in order for data ow to be more widely used. The explicit parallelism of a dataflow program is descriptive and enables an improved utilization of available processing units, however, the independent nodes also implies that some kind of scheduling is required. The need for efficient scheduling becomes even more evident when the number of nodes is larger than the number of processing units and several nodes are running concurrently on one processor core. There exist several data ow models of computation, with different trade-offs between expressiveness and analyzability. These vary from rather restricted but statically schedulable, with minimal scheduling overhead, to dynamic where each ring requires a ring rule to evaluated. The model used in this work, namely RVC-CAL, is a very expressive language, and in the general case it requires dynamic scheduling, however, the strong encapsulation of dataflow nodes enables analysis and the scheduling overhead can be reduced by using quasi-static, or piecewise static, scheduling techniques. The scheduling problem is concerned with nding the few scheduling decisions that must be run-time, while most decisions are pre-calculated. The result is then an, as small as possible, set of static schedules that are dynamically scheduled. To identify these dynamic decisions and to find the concrete schedules, this thesis shows how quasi-static scheduling can be represented as a model checking problem. This involves identifying the relevant information to generate a minimal but complete model to be used for model checking. The model must describe everything that may affect scheduling of the application while omitting everything else in order to avoid state space explosion. This kind of simplification is necessary to make the state space analysis feasible. For the model checker to nd the actual schedules, a set of scheduling strategies are de ned which are able to produce quasi-static schedulers for a wide range of applications. The results of this work show that actor composition with quasi-static scheduling can be used to transform data ow programs to t many different computer architecture with different type and number of cores. This in turn, enables dataflow to provide a more platform independent representation as one application can be fitted to a specific processor architecture without changing the actual program representation. Instead, the program representation is in the context of design space exploration optimized by the development tools to fit the target platform. This work focuses on representing the dataflow scheduling problem as a model checking problem and is implemented as part of a compiler infrastructure. The thesis also presents experimental results as evidence of the usefulness of the approach.

Cohesion Metrics for Improving Software Quality

Abstract Software product metrics aim at measuring the quality of software. Modu- larity is an essential factor in software quality. In this work, metrics related to modularity and especially cohesion of the modules, are considered. The existing metrics are evaluated, and several new alternatives are proposed. The idea of cohesion of modules is that a module or a class should consist of related parts. The closely related principle of coupling says that the relationships between modules should be minimized. First, internal cohesion metrics are considered. The relations that are internal to classes are shown to be useless for quality measurement. Second, we consider external relationships for cohesion. A detailed analysis using design patterns and refactorings confirms that external cohesion is a better quality indicator than internal. Third, motivated by the successes (and problems) of external cohesion metrics, another kind of metric is proposed that represents the quality of modularity of software. This metric can be applied to refactorings related to classes, resulting in a refactoring suggestion system. To describe the metrics formally, a notation for programs is developed. Because of the recursive nature of programming languages, the properties of programs are most compactly represented using grammars and formal lan- guages. Also the tools that were used for metrics calculation are described.

On the Benefits of Abstraction in Concurrent Haskell

Heutzutage haben selbst durchschnittliche Computersysteme mehrere unabhängige Recheneinheiten (Kerne). Wird ein rechenintensives Problem in mehrere Teilberechnungen unterteilt, können diese parallel und damit schneller verarbeitet werden. Obwohl die Entwicklung paralleler Programme mittels Abstraktionen vereinfacht werden kann, ist es selbst für Experten anspruchsvoll, effiziente und korrekte Programme zu schreiben. Während traditionelle Programmiersprachen auf einem eher geringen Abstraktionsniveau arbeiten, bieten funktionale Programmiersprachen wie z.B. Haskell, Möglichkeiten zur fortgeschrittenen Abstrahierung. Das Ziel der vorliegenden Dissertation war es, zu untersuchen, wie gut verschiedene Arten der Abstraktion das Programmieren mit Concurrent Haskell unterstützen. Concurrent Haskell ist eine Bibliothek für Haskell, die parallele Programmierung auf Systemen mit gemeinsamem Speicher ermöglicht. Im Mittelpunkt der Dissertation standen zwei Forschungsfragen. Erstens wurden verschiedene Synchronisierungsansätze verglichen, die sich in ihrem Abstraktionsgrad unterscheiden. Zweitens wurde untersucht, wie Abstraktionen verwendet werden können, um die Komplexität der Parallelisierung vor dem Entwickler zu verbergen. Bei dem Vergleich der Synchronisierungsansätze wurden Locks, Compare-and-Swap Operationen und Software Transactional Memory berücksichtigt. Die Ansätze wurden zunächst bezüglich ihrer Eignung für die Synchronisation einer Prioritätenwarteschlange auf Basis von Skiplists untersucht. Anschließend wurden verschiedene Varianten des Taskpool Entwurfsmusters implementiert (globale Taskpools sowie private Taskpools mit und ohne Taskdiebstahl). Zusätzlich wurde für das Entwurfsmuster eine Abstraktionsschicht entwickelt, welche eine einfache Formulierung von Taskpool-basierten Algorithmen erlaubt. Für die Untersuchung der Frage, ob Haskells Abstraktionsmethoden die Komplexität paralleler Programmierung verbergen können, wurden zunächst stencil-basierte Algorithmen betrachtet. Es wurde eine Bibliothek entwickelt, die eine deklarative Beschreibung von stencil-basierten Algorithmen sowie ihre parallele Ausführung erlaubt. Mit Hilfe dieses deklarativen Interfaces wurde die parallele Implementation vollständig vor dem Anwender verborgen. Anschließend wurde eine eingebettete domänenspezifische Sprache (EDSL) für Knoten-basierte Graphalgorithmen sowie eine entsprechende Ausführungsplattform entwickelt. Die Plattform erlaubt die automatische parallele Verarbeitung dieser Algorithmen. Verschiedene Beispiele zeigten, dass die EDSL eine knappe und dennoch verständliche Formulierung von Graphalgorithmen ermöglicht.

ACTORS: A Model of Concurrent Computation in Distributed Systems

A foundational model of concurrency is developed in this thesis. We examine issues in the design of parallel systems and show why the actor model is suitable for exploiting large-scale parallelism. Concurrency in actors is constrained only by the availability of hardware resources and by the logical dependence inherent in the computation. Unlike dataflow and functional programming, however, actors are dynamically reconfigurable and can model shared resources with changing local state. Concurrency is spawned in actors using asynchronous message-passing, pipelining, and the dynamic creation of actors. This thesis deals with some central issues in distributed computing. Specifically, problems of divergence and deadlock are addressed. For example, actors permit dynamic deadlock detection and removal. The problem of divergence is contained because independent transactions can execute concurrently and potentially infinite processes are nevertheless available for interaction.

A Virtual Machine for a Type-omega Denotational Proof Language

In this thesis, I designed and implemented a virtual machine (VM) for a monomorphic variant of Athena, a type-omega denotational proof language (DPL). This machine attempts to maintain the minimum state required to evaluate Athena phrases. This thesis also includes the design and implementation of a compiler for monomorphic Athena that compiles to the VM. Finally, it includes details on my implementation of a read-eval-print loop that glues together the VM core and the compiler to provide a full, user-accessible interface to monomorphic Athena. The Athena VM provides the same basis for DPLs that the SECD machine does for pure, functional programming and the Warren Abstract Machine does for Prolog.

ADAM: A Decentralized Parallel Computer Architecture Featuring Fast Thread and Data Migration and a Uniform Hardware Abstraction

The furious pace of Moore's Law is driving computer architecture into a realm where the the speed of light is the dominant factor in system latencies. The number of clock cycles to span a chip are increasing, while the number of bits that can be accessed within a clock cycle is decreasing. Hence, it is becoming more difficult to hide latency. One alternative solution is to reduce latency by migrating threads and data, but the overhead of existing implementations has previously made migration an unserviceable solution so far. I present an architecture, implementation, and mechanisms that reduces the overhead of migration to the point where migration is a viable supplement to other latency hiding mechanisms, such as multithreading. The architecture is abstract, and presents programmers with a simple, uniform fine-grained multithreaded parallel programming model with implicit memory management. In other words, the spatial nature and implementation details (such as the number of processors) of a parallel machine are entirely hidden from the programmer. Compiler writers are encouraged to devise programming languages for the machine that guide a programmer to express their ideas in terms of objects, since objects exhibit an inherent physical locality of data and code. The machine implementation can then leverage this locality to automatically distribute data and threads across the physical machine by using a set of high performance migration mechanisms. An implementation of this architecture could migrate a null thread in 66 cycles -- over a factor of 1000 improvement over previous work. Performance also scales well; the time required to move a typical thread is only 4 to 5 times that of a null thread. Data migration performance is similar, and scales linearly with data block size. Since the performance of the migration mechanism is on par with that of an L2 cache, the implementation simulated in my work has no data caches and relies instead on multithreading and the migration mechanism to hide and reduce access latencies.

Implementació d’un sistema xat multiprotocol fent servir el llenguatge de programació ERLANG

Construcció d’una aplicació amb un llenguatge de programació concurrent i distribuït anomenat Erlang. Erlang és un llenguatge de programació funcional amb avaluació estricta, és a dir, assignació única i que inclou una màquina virtual. L’aplicació desenvolupada ha estat la programació d’un xat multiprotocol, el qual s’ha realitzat una primera part que ha consistit en el construcció d’un servidor per la xarxa local i el seu corresponent client. Llavors per poder fer un client més funcional i útil s’ha implementat un altre protocol, IRC. Al tractar-se d’un llenguatge espacialment dissenyat per treballar en processos s’ha intentat dissenyar d’una manera la qual es pugés aprofitar aquesta qualitat

Desarrollo de una aplicación de computador para analizar parámetros estadísticos genético-poblacionales de microsatélites en poblaciones humanas

La información y los datos genéticos que emanan hoy de las investigaciones del genoma humano demandan el desarrollo de herramientas informáticas capaces de procesar la gran cantidad de información disponible. La mayor cantidad de datos genéticos es el resultado de equipos que realizan el análisis simultáneo de cientos o miles de polimorfismos o variaciones genéticas, de nuevas técnicas de laboratorio de mayor rendimiento que, en conjunto, ofrecen una mayor disponibilidad de información en un corto espacio de tiempo. Esta problemática conduce a la necesidad de desarrollar nuevas herramientas informáticas capaces de lidiar con este mayor volumen de datos genéticos. En el caso de la genética de poblaciones, a pesar de que existen herramientas informáticas que permiten procesar y facilitar el análisis de los datos, estas tienen limitaciones como la falta de conocimiento de los usuarios de algunos lenguajes de programación para alimentar la información y otras herramientas informáticas no realizan todas las estimaciones que se requieren y otros presentan limitaciones en cuanto al número de datos que pueden incorporar o manejar. En algunos casos hay redundancia al tener que usarse dos o más herramientas para poder procesar un conjunto de datos de información genética. El presente trabajo tiene por objetivo el desarrollo de una herramienta informática basada en aplicaciones de computador comunes, en este caso Microsoft Excel® y que resuelva todos los problemas y las limitaciones descritas antes. El desarrollo del conjunto de subprogramas que constituyen a Lustro; permiten superar lo anterior, presentar los resultados en un ambiente sencillo, conocido y fácil de operar, simplificando de esta forma el proceso de adaptación del usuario del programa, sin entrenamiento previo, obteniéndose en corto tiempo el procesamiento de la información genética de interés.

Integración de SEXTANTE en GearSCape

SEXTANTE es un marco para el desarrollo de algoritmos dedicados al procesamiento de información geográficamente referenciada, que actualmente cuenta con más de doscientos algoritmos que son capaces de operar sobre datos vectoriales, alfanuméricos y raster. Por otra parte, GearScape es un sistema de información geográfico orientado al geoprocesamiento, que dispone de un lenguaje declarativo que permite el desarrollo de geoprocesos sin necesidad de herramientas de desarrollo complejas. Dicho lenguaje está basado en el estándar SQL y extendido mediante la norma OGC para el acceso a fenómenos simples. Al ser un lenguaje mucho más simple que los lenguajes de programación imperativos (java, .net, python, etc.) la creación de geoprocesos es también más simple, más fácil de documentar, menos propensa a bugs y además la ejecución es optimizada de manera automática mediante el uso de índices y otras técnicas. La posibilidad de describir cadenas de operaciones complejas tiene también valor a modo de documentación: es posible escribir todos los pasos para la resolución de un determinado problema y poder recuperarlo tiempo después, reutilizarlo fácilmente, comunicárselo a otra persona, etc. En definitiva, el lenguaje de geoprocesamiento de GearScape permite "hablar" de geoprocesos. La integración de SEXTANTE en GearScape tiene un doble objetivo. Por una parte se pretende proporcionar la posibilidad de usar cualquiera de los algoritmos con la interfaz habitual de SEXTANTE. Por la otra, se pretende añadir al lenguaje de geoprocesamiento de GearScape la posibilidad de utilizar algoritmos de SEXTANTE. De esta manera, cualquier problema que se resuelva mediante la utilización de varios de estos algoritmes puede ser descrito con el lenguaje de geoprocesamiento de GearScape. A las ventajas del lenguaje de GearScape para la definición de geoprocesos, se añade el abanico de geoprocesos disponible en SEXTANTE, por lo que el lenguaje de geoprocesamiento de GearScape nos permite "hablar" utilizando vocabulario de SEXTANTE

Student's watcher : comparison between ASP.NET and PHP+CSS

"Student’s Watcher” is a small Web application which wants to show in a visual, simple and fast way, the evolution of the students. The main project table displays such things as marks and comments about students. We can add a comment for each mark to explain why this mark. The objective is to be able to know if some student has a problem, how is going his year, marks in other courses, or even, to know if he has a bad week in a different subjects. We can see the evolution of students in past years to do an objective comparison. It also allows inserting global comments of student, we have a list of these, and all professors can add new ones, where we can see more general valuations. “Student’s Watcher” was begun in ASP.net, but finally my project would be developed in PHP, HTML and CSS. This project wants to be a comparison between two of most important languages used nowadays, ASPX and PHP

Running Climate Models On The NERC Cluster Grid Using G-Rex

Compute grids are used widely in many areas of environmental science, but there has been limited uptake of grid computing by the climate modelling community, partly because the characteristics of many climate models make them difficult to use with popular grid middleware systems. In particular, climate models usually produce large volumes of output data, and running them usually involves complicated workflows implemented as shell scripts. For example, NEMO (Smith et al. 2008) is a state-of-the-art ocean model that is used currently for operational ocean forecasting in France, and will soon be used in the UK for both ocean forecasting and climate modelling. On a typical modern cluster, a particular one year global ocean simulation at 1-degree resolution takes about three hours when running on 40 processors, and produces roughly 20 GB of output as 50000 separate files. 50-year simulations are common, during which the model is resubmitted as a new job after each year. Running NEMO relies on a set of complicated shell scripts and command utilities for data pre-processing and post-processing prior to job resubmission. Grid Remote Execution (G-Rex) is a pure Java grid middleware system that allows scientific applications to be deployed as Web services on remote computer systems, and then launched and controlled as if they are running on the user's own computer. Although G-Rex is general purpose middleware it has two key features that make it particularly suitable for remote execution of climate models: (1) Output from the model is transferred back to the user while the run is in progress to prevent it from accumulating on the remote system and to allow the user to monitor the model; (2) The client component is a command-line program that can easily be incorporated into existing model work-flow scripts. G-Rex has a REST (Fielding, 2000) architectural style, which allows client programs to be very simple and lightweight and allows users to interact with model runs using only a basic HTTP client (such as a Web browser or the curl utility) if they wish. This design also allows for new client interfaces to be developed in other programming languages with relatively little effort. The G-Rex server is a standard Web application that runs inside a servlet container such as Apache Tomcat and is therefore easy to install and maintain by system administrators. G-Rex is employed as the middleware for the NERC1 Cluster Grid, a small grid of HPC2 clusters belonging to collaborating NERC research institutes. Currently the NEMO (Smith et al. 2008) and POLCOMS (Holt et al, 2008) ocean models are installed, and there are plans to install the Hadley Centre’s HadCM3 model for use in the decadal climate prediction project GCEP (Haines et al., 2008). The science projects involving NEMO on the Grid have a particular focus on data assimilation (Smith et al. 2008), a technique that involves constraining model simulations with observations. The POLCOMS model will play an important part in the GCOMS project (Holt et al, 2008), which aims to simulate the world’s coastal oceans. A typical use of G-Rex by a scientist to run a climate model on the NERC Cluster Grid proceeds as follows :(1) The scientist prepares input files on his or her local machine. (2) Using information provided by the Grid’s Ganglia3 monitoring system, the scientist selects an appropriate compute resource. (3) The scientist runs the relevant workflow script on his or her local machine. This is unmodified except that calls to run the model (e.g. with “mpirun”) are simply replaced with calls to "GRexRun" (4) The G-Rex middleware automatically handles the uploading of input files to the remote resource, and the downloading of output files back to the user, including their deletion from the remote system, during the run. (5) The scientist monitors the output files, using familiar analysis and visualization tools on his or her own local machine. G-Rex is well suited to climate modelling because it addresses many of the middleware usability issues that have led to limited uptake of grid computing by climate scientists. It is a lightweight, low-impact and easy-to-install solution that is currently designed for use in relatively small grids such as the NERC Cluster Grid. A current topic of research is the use of G-Rex as an easy-to-use front-end to larger-scale Grid resources such as the UK National Grid service.

Depth to mate and the 50-move rule

The most popular endgame tables (EGTs) documenting ‘DTM’ Depth to Mate in chess endgames are those of Eugene Nalimov but these do not recognise the FIDE 50-move rule ‘50mr’. This paper marks the creation by the first author of EGTs for sub-6-man (s6m) chess and beyond which give DTM as affected by the ply count pc. The results are put into the context of previous work recognising the 50mr and are compared with the original unmoderated DTM results. The work is also notable for being the first EGT generation work to use the functional programming language HASKELL.