An Exploration of the challenges associated with software logging in large systems

Thesis (Master, Computing) -- Queen's University, 2016-05-29 18:11:34.114

An Exploration of the challenges associated with software logging in large systems

Over the past few years, logging has evolved from from simple printf statements to more complex and widely used logging libraries. Today logging information is used to support various development activities such as fixing bugs, analyzing the results of load tests, monitoring performance and transferring knowledge. Recent research has examined how to improve logging practices by informing developers what to log and where to log. Furthermore, the strong dependence on logging has led to the development of logging libraries that have reduced the intricacies of logging, which has resulted in an abundance of log information. Two recent challenges have emerged as modern software systems start to treat logging as a core aspect of their software. In particular, 1) infrastructural challenges have emerged due to the plethora of logging libraries available today and 2) processing challenges have emerged due to the large number of log processing tools that ingest logs and produce useful information from them. In this thesis, we explore these two challenges. We first explore the infrastructural challenges that arise due to the plethora of logging libraries available today. As systems evolve, their logging infrastructure has to evolve (commonly this is done by migrating to new logging libraries). We explore logging library migrations within Apache Software Foundation (ASF) projects. We i find that close to 14% of the pro jects within the ASF migrate their logging libraries at least once. For processing challenges, we explore the different factors which can affect the likelihood of a logging statement changing in the future in four open source systems namely ActiveMQ, Camel, Cloudstack and Liferay. Such changes are likely to negatively impact the log processing tools that must be updated to accommodate such changes. We find that 20%-45% of the logging statements within the four systems are changed at least once. We construct random forest classifiers and Cox models to determine the likelihood of both just-introduced and long-lived logging statements changing in the future. We find that file ownership, developer experience, log density and SLOC are important factors in determining the stability of logging statements.

Software for Designing Custom Orbital-Craniofacial Implant Plans

Purpose: Custom cranio-orbital implants have been shown to achieve better performance than their hand-shaped counterparts by restoring skull anatomy more accurately and by reducing surgery time. Designing a custom implant involves reconstructing a model of the patient's skull using their computed tomography (CT) scan. The healthy side of the skull model, contralateral to the damaged region, can then be used to design an implant plan. Designing implants for areas of thin bone, such as the orbits, is challenging due to poor CT resolution of bone structures. This makes preoperative design time-intensive since thin bone structures in CT data must be manually segmented. The objective of this thesis was to research methods to accurately and efficiently design cranio-orbital implant plans, with a focus on the orbits, and to develop software that integrates these methods. Methods: The software consists of modules that use image and surface restoration approaches to enhance both the quality of CT data and the reconstructed model. It enables users to input CT data, and use tools to output a skull model with restored anatomy. The skull model can then be used to design the implant plan. The software was designed using 3D Slicer, an open-source medical visualization platform. It was tested on CT data from thirteen patients. Results: The average time it took to create a skull model with restored anatomy using our software was 0.33 hours ± 0.04 STD. In comparison, the design time of the manual segmentation method took between 3 and 6 hours. To assess the structural accuracy of the reconstructed models, CT data from the thirteen patients was used to compare the models created using our software with those using the manual method. When registering the skull models together, the difference between each set of skulls was found to be 0.4 mm ± 0.16 STD. Conclusions: We have developed a software to design custom cranio-orbital implant plans, with a focus on thin bone structures. The method described decreases design time, and is of similar accuracy to the manual method.

Dashboard per monitorare lo sviluppo software: criticità, vantaggi

Lo scopo di questa tesi e studiare l’uso di ”cruscotti” (in inglese Dashboard) per il monitoraggio dello sviluppo software, approfondendo i metodi di raccolta delle metriche e come esse vengono gestite. Nello specifico, analizzerò l’ambiente di sviluppo Compositional Agile System (CAS), sviluppando un nuovo plugin per l’IDE Microsoft Visual Studio Code, che e open source. Verranno proposti nuovi metodi di implementazione e utilizzo delle Dashboard e possibili miglioramenti dell’ambiente CAS.

Progettazione e sviluppo di uno strumento per la scansione di progetti software alla ricerca di potenziali segni di plagio

Questa tesi nasce dalla necessità di sviluppare uno strumento capace d’individuare potenziali plagi in progetti software. Il plagiarismo nel software è pratica che, nel tempo, ha visto numerosi scontri legali (ad esempio, Oracle contro Google per Android), e per il quale sono pochi i progetti open source di facile utilizzo pratico. L’elaborato, quindi, si addentra nelle tecniche di analisi e d’individuazione di possibili plagi, presentando il processo di progettazione e sviluppo dello strumento.

Progetto e implementazione di software di segmentazione assistita di immagini mediche e predizione di biomarkers

La segmentazione prevede la partizione di un'immagine in aree strutturalmente o semanticamente coerenti. Nell'imaging medico, è utilizzata per identificare, contornandole, Regioni di Interesse (ROI) clinico, quali lesioni tumorali, oggetto di approfondimento tramite analisi semiautomatiche e automatiche, o bersaglio di trattamenti localizzati. La segmentazione di lesioni tumorali, assistita o automatica, consiste nell’individuazione di pixel o voxel, in immagini o volumi, appartenenti al tumore. La tecnica assistita prevede che il medico disegni la ROI, mentre quella automatica è svolta da software addestrati, tra cui i sistemi Computer Aided Detection (CAD). Mediante tecniche di visione artificiale, dalle ROI si estraggono caratteristiche numeriche, feature, con valore diagnostico, predittivo, o prognostico. L’obiettivo di questa Tesi è progettare e sviluppare un software di segmentazione assistita che permetta al medico di disegnare in modo semplice ed efficace una o più ROI in maniera organizzata e strutturata per futura elaborazione ed analisi, nonché visualizzazione. Partendo da Aliza, applicativo open-source, visualizzatore di esami radiologici in formato DICOM, è stata estesa l’interfaccia grafica per gestire disegno, organizzazione e memorizzazione automatica delle ROI. Inoltre, è stata implementata una procedura automatica di elaborazione ed analisi di ROI disegnate su lesioni tumorali prostatiche, per predire, di ognuna, la probabilità di cancro clinicamente non-significativo e significativo (con prognosi peggiore). Per tale scopo, è stato addestrato un classificatore lineare basato su Support Vector Machine, su una popolazione di 89 pazienti con 117 lesioni (56 clinicamente significative), ottenendo, in test, accuratezza = 77%, sensibilità = 86% e specificità = 69%. Il sistema sviluppato assiste il radiologo, fornendo una seconda opinione, non vincolante, adiuvante nella definizione del quadro clinico e della prognosi, nonché delle scelte terapeutiche.

Analysing Road Barrier using GIS Software: The Case Study of Bologna

The direct attention of this thesis is the maintenance of road elements to improve road safety. The goal of the research is to prioritise maintenance for barriers based on factors such as the terrain of the site, deformations, degradation of the components, and adherence to the original installation. Using these factors, a coefficient is calculated to determine the maintenance priority for each barrier. To ease understanding and visualisation, data was uploaded and processed in a GIS environment to generate analysis and maps. This was done using GIS, a free and open-source GIS software. Information about the features of the barriers was collected through both on-site and online examination. During on-site inspections, a database of geotagged photos was created to aid in the survey. GIS capabilities word fully utilised by using geoprocessing tools for more in-depth analysis.

Feature selection environment for genomic applications

Background: Feature selection is a pattern recognition approach to choose important variables according to some criteria in order to distinguish or explain certain phenomena (i.e., for dimensionality reduction). There are many genomic and proteomic applications that rely on feature selection to answer questions such as selecting signature genes which are informative about some biological state, e. g., normal tissues and several types of cancer; or inferring a prediction network among elements such as genes, proteins and external stimuli. In these applications, a recurrent problem is the lack of samples to perform an adequate estimate of the joint probabilities between element states. A myriad of feature selection algorithms and criterion functions have been proposed, although it is difficult to point the best solution for each application. Results: The intent of this work is to provide an open-source multiplataform graphical environment for bioinformatics problems, which supports many feature selection algorithms, criterion functions and graphic visualization tools such as scatterplots, parallel coordinates and graphs. A feature selection approach for growing genetic networks from seed genes ( targets or predictors) is also implemented in the system. Conclusion: The proposed feature selection environment allows data analysis using several algorithms, criterion functions and graphic visualization tools. Our experiments have shown the software effectiveness in two distinct types of biological problems. Besides, the environment can be used in different pattern recognition applications, although the main concern regards bioinformatics tasks.

Saberes em gest??o p??blica: uma pr??tica de gest??o do conhecimento para as organiza????es

Neste paper, relata-se uma pr??tica de gest??o do conhecimento da Escola de Governo do Paran??: o SabeRES em Gest??o P??blica, um reposit??rio institucional de acesso livre que disponibiliza na Internet toda a produ????o t??cnica e cient??fica da Escola de Governo do Paran?? e de outras organiza????es de aprendizagem. Inicialmente discute-se a defini????o de conte??do e conhecimento, passando ao marco legal dos reposit??rios institucionais, com o Movimento Mundial Open Access. Em seguida, discute-se a tecnologia de software livre na cria????o de reposit??rios e as suas caracter??sticas e quais os sistemas de gest??o de conte??dos mais utilizados em open source. Na seq????ncia, apresenta-se a experi??ncia da Escola de Governo do Paran?? nas tr??s fases do SabeRES: na cria????o, na gest??o e na publica????o dos documentos. A metodologia utilizada constou de uma an??lise SWOT, seguida de planejamento gr??fico e estrutural, defini????o da arquitetura, constru????o da interface gr??fica com ferramenta XOOPS (software livre) e defini????o da ??rvore inicial de menus. O treinamento da equipe para a gest??o e manuten????o do reposit??rio foi imprescind??vel para a inser????o dos documentos e a disponibiliza????o do sistema na Internet. Por fim, uma C??mara T??cnica foi constitu??da para a aprecia????o dos materiais recebidos e para dar contribui????es de melhoria na gest??o do SabeRES.

Towards a Catalog of Usability Smells

This paper presents a catalog of smells in the context of interactive applications. These so-called usability smells are indicators of poor design on an application’s user interface, with the potential to hinder not only its usability but also its maintenance and evolution. To eliminate such usability smells we discuss a set of program/usability refactorings. In order to validate the presented usability smells catalog, and the associated refactorings, we present a preliminary empirical study with software developers in the context of a real open source hospital management application. Moreover, a tool that computes graphical user interface behavior models, giving the applications’ source code, is used to automatically detect usability smells at the model level.

Framework genérica de business intelligence para suporte financeiro

Actualmente, não existem ferramentas open source de Business Intelligence (BI) para suporte à gestão e análise financeira nas empresas, de acordo com o sistema de normalização contabilística (SNC). As diferentes características de cada negócio, juntamente com os requisitos impostos pelo SNC, tornam complexa a criação de uma Framework financeira genérica, que satisfaça, de forma eficiente, as análises financeiras necessárias à gestão das empresas. O objectivo deste projecto é propor uma framework baseada em OLAP, capaz de dar suporte à gestão contabilística e análise financeira, recorrendo exclusivamente a software open source na sua implementação, especificamente, a plataforma Pentaho. Toda a informação contabilística, obtida através da contabilidade geral, da contabilidade analítica, da gestão orçamental e da análise financeira é armazenada num Data mart. Este Data mart suportará toda a análise financeira, incluindo a análise de desvios orçamentais e de fluxo de capitais, permitindo às empresas ter uma ferramenta de BI, compatível com o SNC, que as ajude na tomada de decisões.

Visualização de percursos em dispositivos móveis

Cada vez mais as pessoas se deslocam para locais desconhecidos por motivos profissionais, financeiros ou simplesmente por lazer. Em todas estas situações as primeiras dificuldades com que se deparam são saber qual o percurso a adoptar para chegar a um determinado destino e a falta de informação sobre o local onde se encontram, especialmente sobre os possíveis pontos de interesse que existem em redor. O presente trabalho propõe a concepção e a implementação de um sistema de visualização para dispositivos móveis de percursos pré-definidos baseado em standards. Pretende-se construir um software aberto de modo a poder ser utilizado em diversas aplicações geográficas modulares e facilitar a investigação na área, pois a maior parte do software disponibilizado é proprietário havendo apenas hipótese de interagir através de API disponibilizada. O trabalho pretende servir utilizadores que pretendam tirar partido de um guia digital móvel, materializado num dispositivo móvel com capacidade visualização gráfica do percurso. Pode ser utilizado nas mais diversas situações: orientação em trilhos ou rotas turísticas; identificação e orientação de pontos de interesse numa cidade ou região; ou informação de contexto. Para estimular a atenção do utilizador é proposto um mecanismo de alertas/avisos que serão despoletados durante a execução do percurso. Como muitas das funcionalidades disponibilizadas pelo sistema necessitam de efectuar cálculo de distâncias, é proposto um modelo de determinação de distâncias. De modo a gerir o conteúdo dos ficheiros utilizados é proposto um mecanismo de processamento de ficheiros, que permite efectuar transformações, validações e gravações de ficheiros. Para além da concepção e correspondente implementação do Visualizador, este trabalho apresenta ainda cenários de aplicação do sistema, ilustrando consequentemente a sua utilização em situações reais e encontra-se disponível para ser usado e melhorado pela comunidade científica, pois foi feito o registo como open source no site sourceforge.net.

LightStream: sistema de comunicação Publish/Subscribe com WebSockets

A realização deste trabalho permitiu disponibilizar uma plataforma de caracter inovador, que consegue colmatar muitos problemas existentes hoje em dia, na maior parte das aplicações Web, não só a nível de utilização eficiente de largura de banda, bem como a nível de interoperabilidade entre tecnologias. Devido às características cross-platform da plataforma, é possível realizar comunicação de baixa latência entre sistemas desenvolvidos em tecnologias completamente diferentes, como é o caso de Javascript e .NET. A implementação desta plataforma representou não só um desafio a nível de engenharia de software, mas também um desafio tecnológico, tendo em conta o número de tecnologias diferentes que são utilizadas nas diversas camadas. Neste trabalho é também disponibilizada uma Framework de comunicação assíncrona, cross-platform, que permite integrar componentes de várias tecnologias diferentes, como é o caso de NodeJS e .NET, através da exposição de serviços com o mesmo tipo de interfaces utilizados em WCF. A implementação desta Framework representou um grande desafio e poderá vir a ser evoluída no futuro, como projeto open source.

Study on influence of the new CoDeSys V3.0 on the existing CoDeSys discipline of the Mind8 Engineering Center

CoDeSys "Controller Development Systems" is a development environment for programming in the area of automation controllers. It is an open source solution completely in line with the international industrial standard IEC 61131-3. All five programming languages for application programming as defined in IEC 61131-3 are available in the development environment. These features give professionals greater flexibility with regard to programming and allow control engineers have the ability to program for many different applications in the languages in which they feel most comfortable. Over 200 manufacturers of devices from different industrial sectors offer intelligent automation devices with a CoDeSys programming interface. In 2006, version 3 was released with new updates and tools. One of the great innovations of the new version of CoDeSys is object oriented programming. Object oriented programming (OOP) offers great advantages to the user for example when wanting to reuse existing parts of the application or when working on one application with several developers. For this reuse can be prepared a source code with several well known parts and this is automatically generated where necessary in a project, users can improve then the time/cost/quality management. Until now in version 2 it was necessary to have hardware interface called “Eni-Server” to have access to the generated XML code. Another of the novelties of the new version is a tool called Export PLCopenXML. This tool makes it possible to export the open XML code without the need of specific hardware. This type of code has own requisites to be able to comply with the standard described above. With XML code and with the knowledge how it works it is possible to do component-oriented development of machines with modular programming in an easy way. Eplan Engineering Center (EEC) is a software tool developed by Mind8 GmbH & Co. KG that allows configuring and generating automation projects. Therefore it uses modules of PLC code. The EEC already has a library to generate code for CoDeSys version 2. For version 3 and the constant innovation of drivers by manufacturers, it is necessary to implement a new library in this software. Therefore it is important to study the XML export to be then able to design any type of machine. The purpose of this master thesis is to study the new version of the CoDeSys XML taking into account all aspects and impact on the existing CoDeSys V2 models and libraries in the company Harro Höfliger Verpackungsmaschinen GmbH. For achieve this goal a small sample named “Traffic light” in CoDeSys version 2 will be done and then, using the tools of the new version it there will be a project with version 3 and also the EEC implementation for the automatically generated code.

Arrow Plot: a new graphical tool for selecting up and down regulated genes and genes differentially expressed on samples subgroups

Background: A common task in analyzing microarray data is to determine which genes are differentially expressed across two (or more) kind of tissue samples or samples submitted under experimental conditions. Several statistical methods have been proposed to accomplish this goal, generally based on measures of distance between classes. It is well known that biological samples are heterogeneous because of factors such as molecular subtypes or genetic background that are often unknown to the experimenter. For instance, in experiments which involve molecular classification of tumors it is important to identify significant subtypes of cancer. Bimodal or multimodal distributions often reflect the presence of subsamples mixtures. Consequently, there can be genes differentially expressed on sample subgroups which are missed if usual statistical approaches are used. In this paper we propose a new graphical tool which not only identifies genes with up and down regulations, but also genes with differential expression in different subclasses, that are usually missed if current statistical methods are used. This tool is based on two measures of distance between samples, namely the overlapping coefficient (OVL) between two densities and the area under the receiver operating characteristic (ROC) curve. The methodology proposed here was implemented in the open-source R software. Results: This method was applied to a publicly available dataset, as well as to a simulated dataset. We compared our results with the ones obtained using some of the standard methods for detecting differentially expressed genes, namely Welch t-statistic, fold change (FC), rank products (RP), average difference (AD), weighted average difference (WAD), moderated t-statistic (modT), intensity-based moderated t-statistic (ibmT), significance analysis of microarrays (samT) and area under the ROC curve (AUC). On both datasets all differentially expressed genes with bimodal or multimodal distributions were not selected by all standard selection procedures. We also compared our results with (i) area between ROC curve and rising area (ABCR) and (ii) the test for not proper ROC curves (TNRC). We found our methodology more comprehensive, because it detects both bimodal and multimodal distributions and different variances can be considered on both samples. Another advantage of our method is that we can analyze graphically the behavior of different kinds of differentially expressed genes. Conclusion: Our results indicate that the arrow plot represents a new flexible and useful tool for the analysis of gene expression profiles from microarrays.