Machine learning as a service for high energy physics (MLaaS4HEP): a service for ML-based data analyses

With the CERN LHC program underway, there has been an acceleration of data growth in the High Energy Physics (HEP) field and the usage of Machine Learning (ML) in HEP will be critical during the HL-LHC program when the data that will be produced will reach the exascale. ML techniques have been successfully used in many areas of HEP nevertheless, the development of a ML project and its implementation for production use is a highly time-consuming task and requires specific skills. Complicating this scenario is the fact that HEP data is stored in ROOT data format, which is mostly unknown outside of the HEP community. The work presented in this thesis is focused on the development of a ML as a Service (MLaaS) solution for HEP, aiming to provide a cloud service that allows HEP users to run ML pipelines via HTTP calls. These pipelines are executed by using the MLaaS4HEP framework, which allows reading data, processing data, and training ML models directly using ROOT files of arbitrary size from local or distributed data sources. Such a solution provides HEP users non-expert in ML with a tool that allows them to apply ML techniques in their analyses in a streamlined manner. Over the years the MLaaS4HEP framework has been developed, validated, and tested and new features have been added. A first MLaaS solution has been developed by automatizing the deployment of a platform equipped with the MLaaS4HEP framework. Then, a service with APIs has been developed, so that a user after being authenticated and authorized can submit MLaaS4HEP workflows producing trained ML models ready for the inference phase. A working prototype of this service is currently running on a virtual machine of INFN-Cloud and is compliant to be added to the INFN Cloud portfolio of services.

Sviluppo di un ambiente virtualizzato per la simulazione di attacchi a reti time-sensitive

Questo lavoro di tesi ha visto come obiettivo finale quello di realizzare una se- rie di attacchi, alcuni di questi totalmente originali, ai protocolli della famiglia Time-Sensitive Networking (TSN) attraverso lo sviluppo di un’infrastruttura virtualizzata. L’infrastruttura è stata costruita e progettata utilizzando mac- chine virtuali con Quick EMUlator (QEMU) come strato di virtualizzazione ed accelerate attraverso Kernel-based Virtual Machine (KVM). Il progetto è stato concepito come Infrastrucutre as Code (IaC), attraverso l’ausilio di Ansible e alcuni script shell utilizzati come collante per le varie parti del progetto.

Dynamic power- and failure-aware cloud resources allocation for Sets of independent tasks

Cloud computing is increasingly being adopted in different scenarios, like social networking, business applications, scientific experiments, etc. Relying in virtualization technology, the construction of these computing environments targets improvements in the infrastructure, such as power-efficiency and fulfillment of users’ SLA specifications. The methodology usually applied is packing all the virtual machines on the proper physical servers. However, failure occurrences in these networked computing systems can induce substantial negative impact on system performance, deviating the system from ours initial objectives. In this work, we propose adapted algorithms to dynamically map virtual machines to physical hosts, in order to improve cloud infrastructure power-efficiency, with low impact on users’ required performance. Our decision making algorithms leverage proactive fault-tolerance techniques to deal with systems failures, allied with virtual machine technology to share nodes resources in an accurately and controlled manner. The results indicate that our algorithms perform better targeting power-efficiency and SLA fulfillment, in face of cloud infrastructure failures.

Evaluating Android OS for embedded real-time systems

Since its official public release, Android has captured the interest from companies, developers and the general audience. From that time up to now, this software platform has been constantly improved either in terms of features or supported hardware and, at the same time, extended to new types of devices different from the originally intended mobile ones. However, there is a feature that has not been explored yet - its real-time capabilities. This paper intends to explore this gap and provide a basis for discussion on the suitability of Android in order to be used in Open Real-Time environments. By analysing the software platform, with the main focus on the virtual machine and its underlying operating system environments, we are able to point out its current limitations and, therefore, provide a hint on different perspectives of directions in order to make Android suitable for these environments. It is our position that Android may provide a suitable architecture for real-time embedded systems, but the real-time community should address its limitations in a joint effort at all of the platform layers.

PIASA: A power and interference aware resource management strategy for heterogeneous workloads in cloud data centers

Cloud data centers have been progressively adopted in different scenarios, as reflected in the execution of heterogeneous applications with diverse workloads and diverse quality of service (QoS) requirements. Virtual machine (VM) technology eases resource management in physical servers and helps cloud providers achieve goals such as optimization of energy consumption. However, the performance of an application running inside a VM is not guaranteed due to the interference among co-hosted workloads sharing the same physical resources. Moreover, the different types of co-hosted applications with diverse QoS requirements as well as the dynamic behavior of the cloud makes efficient provisioning of resources even more difficult and a challenging problem in cloud data centers. In this paper, we address the problem of resource allocation within a data center that runs different types of application workloads, particularly CPU- and network-intensive applications. To address these challenges, we propose an interference- and power-aware management mechanism that combines a performance deviation estimator and a scheduling algorithm to guide the resource allocation in virtualized environments. We conduct simulations by injecting synthetic workloads whose characteristics follow the last version of the Google Cloud tracelogs. The results indicate that our performance-enforcing strategy is able to fulfill contracted SLAs of real-world environments while reducing energy costs by as much as 21%.

Navegação web anónima e segura

A Web aproximou a humanidade dos seus pares a um nível nunca antes visto. Com esta facilidade veio também o cibercrime, o terrorismo e outros fenómenos característicos de uma sociedade tecnológica, plenamente informatizada e onde as fronteiras terrestres pouco importam na limitação dos agentes ativos, nocivos ou não, deste sistema. Recentemente descobriu-se que as grandes nações “vigiam” atentamente os seus cidadãos, desrespeitando qualquer limite moral e tecnológico, podendo escutar conversas telefónicas, monitorizar o envio e receção de e-mails, monitorizar o tráfego Web do cidadão através de poderosíssimos programas de monitorização e vigilância. Noutros cantos do globo, nações em tumulto ou envoltas num manto da censura perseguem os cidadãos negando-lhes o acesso à Web. Mais mundanamente, há pessoas que coagem e invadem a privacidade de conhecidos e familiares, vasculhando todos os cantos dos seus computadores e hábitos de navegação. Neste sentido, após o estudo das tecnologias que permitem a vigilância constante dos utilizadores da Web, foram analisadas soluções que permitem conceder algum anónimato e segurança no tráfego Web. Para suportar o presente estudo, foi efetuada uma análise das plataformas que permitem uma navegação anónima e segura e um estudo das tecnologias e programas com potencial de violação de privacidade e intrusão informática usados por nações de grande notoriedade. Este trabalho teve como objetivo principal analisar as tecnologias de monitorização e de vigilância informática identificando as tecnologias disponíveis, procurando encontrar potenciais soluções no sentido de investigar a possibilidade de desenvolver e disponibilizar uma ferramenta multimédia alicerçada em Linux e em LiveDVD (Sistema Operativo Linux que corre a partir do DVD sem necessidade de instalação). Foram integrados recursos no protótipo com o intuito de proporcionar ao utilizador uma forma ágil e leiga para navegar na Web de forma segura e anónima, a partir de um sistema operativo (SO) virtualizado e previamente ajustado para o âmbito anteriormente descrito. O protótipo foi testado e avaliado por um conjunto de cidadãos no sentido de aferir o seu potencial. Termina-se o documento com as conclusões e o trabalho a desenvolver futuramente.

Dependable decentralized storage management for cloud computing

The MAP-i Doctoral Program of the Universities of Minho, Aveiro and Porto.

Sparse nonlinear discriminants

Kernel-Functions, Machine Learning, Least Squares, Speech Recognition, Classification, Regression

Gestión sostenible de clústers de recursos virtuales

Este proyecto tiene como finalidad ofrecer un servicio de computación en forma de máquina virtual, utilizando los recursos internos de Atos Research & Innovation. Además, se pretende implementar este servicio sobre os excendentes de máquinas del propio departamento. La prestación de este servicio se realiza mediante un gestor de la infraestructura de forma centralizada. Para la implantación de este entorno se ha definido la adopción en fases y profundizado en dos de los gestores más activos en la investigación del modelo Cloud Computing (Open Nebula, Eucalyptus).

An efficient use of virtualization in grid/cloud environments

Grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational resources. Grid enables access to the resources but it does not guarantee any quality of service. Moreover, Grid does not provide performance isolation; job of one user can influence the performance of other user’s job. The other problem with Grid is that the users of Grid belong to scientific community and the jobs require specific and customized software environment. Providing the perfect environment to the user is very difficult in Grid for its dispersed and heterogeneous nature. Though, Cloud computing provide full customization and control, but there is no simple procedure available to submit user jobs as in Grid. The Grid computing can provide customized resources and performance to the user using virtualization. A virtual machine can join the Grid as an execution node. The virtual machine can also be submitted as a job with user jobs inside. Where the first method gives quality of service and performance isolation, the second method also provides customization and administration in addition. In this thesis, a solution is proposed to enable virtual machine reuse which will provide performance isolation with customization and administration. The same virtual machine can be used for several jobs. In the proposed solution customized virtual machines join the Grid pool on user request. Proposed solution describes two scenarios to achieve this goal. In first scenario, user submits their customized virtual machine as a job. The virtual machine joins the Grid pool when it is powered on. In the second scenario, user customized virtual machines are preconfigured in the execution system. These virtual machines join the Grid pool on user request. Condor and VMware server is used to deploy and test the scenarios. Condor supports virtual machine jobs. The scenario 1 is deployed using Condor VM universe. The second scenario uses VMware-VIX API for scripting powering on and powering off of the remote virtual machines. The experimental results shows that as scenario 2 does not need to transfer the virtual machine image, the virtual machine image becomes live on pool more faster. In scenario 1, the virtual machine runs as a condor job, so it easy to administrate the virtual machine. The only pitfall in scenario 1 is the network traffic.

Monitor de servidors JMS

El monitor de servidors JMS és un projecte basat en el disseny i implementacio d'una eina GUI, destinada a programadors i equips de proves que treballin amb la tecnología Java Message Service, multiplataforma i multiservidor, que podrà monitoritzar un nombre variat de servidors JMS des de qualsevol sistema que tingui una màquina virtual de Java instal·lada. L'aplicació té com a principal objectiu visualitzar de forma clara i senzilla l'estat global d'un servidor JMS, mostrant les cues i tòpics creats, juntament amb la possibilitat de realitzar accions sobre les mateixes destinacions (enviament i eliminació de missatges residents al servidor) i la creació de gràfiques sobre el tràfic de missatges.

Desarrollo de aplicaciones móviles J2ME : BloobsME

Construcció d'un programari d'entreteniment per a dispositius mòbils que puguin executar una màquina virtual Java J2ME.

Estudi comparatiu de la gestió de la memòria en dues màquines virtuals

Aquest document presenta l'arquitectura de la màquina virtual Java (Java Virtual Machine, o JVM). Java és un llenguatge de programació cada vegada més extès i, per tant, més utilitzat. El seu àmbit d¿execució s¿estèn actualment per qualsevol plataforma, des de un dispositiu mòbil (telèfon, PDA...) com un mainframe (com per exemple, el model S/390 sota z/OS de IBM).

Server Virtualization

Virtualisoinnin ideana on kuvata tietotekniikkaan liittyvät laiteresurssit ryhminä. Kun jonkin tehtävän suoritukseen tarvitaan resursseja, ne kerätään erikseen jokaisesta ryhmästä. Virtualisoinnin yksi osa-alue on palvelimen tai palvelinten virtualisointi, jossa pyritään hyödyntämään palvelinlaitteisto mahdollisimman tehokkaasti. Tehokkuus saavutetaan käyttämällä erillisiä instansseja, joita kutsutaan virtuaalikoneiksi. Tässä diplomityössä esitellään ja verrataan erilaisia palvelinten virtualisointimalleja ja tekniikoita, joita voidaan käyttää IA-32 arkkitehtuurin kanssa. Eroa virtualisoinnin ja eri partitiointitekniikoiden välillä tarkastellaan erikseen. Lisäksi muutoksia, joita palvelinten virtualisointi aiheuttaa infrastruktuuriin, ympäristöön ja laitteistoon käsitellään yleisellä tasolla. Teorian oikeellisuutta todistettiin suorittamalla useita testejä käyttäen kahta eri virtualisointiohjelmistoa. Testien perusteella palvelinten virtualisointi vähentää suorituskykyä ja luo ympäristön, jonka hallitseminen on vaikeampaa verrattuna perinteiseen ympäristöön. Myös tietoturvaa on katsottava uudesta näkökulmasta, sillä fyysistä eristystä ei virtuaalikoneille voida toteuttaa. Jotta virtualisoinnista saataisiin mahdollisimman suuri hyöty tuotantoympäristössä, vaaditaan tarkkaa harkintaa ja suunnitelmallisuutta. Parhaat käyttökohteet ovat erilaiset testiympäristöt, joissa vaatimukset suorituskyvyn ja turvallisuuden suhteen eivät ole niin tarkat.

A Co-Processor Approach for Efficient Java Execution in Embedded Systems

This thesis deals with a hardware accelerated Java virtual machine, named REALJava. The REALJava virtual machine is targeted for resource constrained embedded systems. The goal is to attain increased computational performance with reduced power consumption. While these objectives are often seen as trade-offs, in this context both of them can be attained simultaneously by using dedicated hardware. The target level of the computational performance of the REALJava virtual machine is initially set to be as fast as the currently available full custom ASIC Java processors. As a secondary goal all of the components of the virtual machine are designed so that the resulting system can be scaled to support multiple co-processor cores. The virtual machine is designed using the hardware/software co-design paradigm. The partitioning between the two domains is flexible, allowing customizations to the resulting system, for instance the floating point support can be omitted from the hardware in order to decrease the size of the co-processor core. The communication between the hardware and the software domains is encapsulated into modules. This allows the REALJava virtual machine to be easily integrated into any system, simply by redesigning the communication modules. Besides the virtual machine and the related co-processor architecture, several performance enhancing techniques are presented. These include techniques related to instruction folding, stack handling, method invocation, constant loading and control in time domain. The REALJava virtual machine is prototyped using three different FPGA platforms. The original pipeline structure is modified to suit the FPGA environment. The performance of the resulting Java virtual machine is evaluated against existing Java solutions in the embedded systems field. The results show that the goals are attained, both in terms of computational performance and power consumption. Especially the computational performance is evaluated thoroughly, and the results show that the REALJava is more than twice as fast as the fastest full custom ASIC Java processor. In addition to standard Java virtual machine benchmarks, several new Java applications are designed to both verify the results and broaden the spectrum of the tests.