Development of a heterogeneous clustering algorithm for particle shower reconstruction in high energy physics using the SYCL abstraction layer

Nei prossimi anni è atteso un aggiornamento sostanziale di LHC, che prevede di aumentare la luminosità integrata di un fattore 10 rispetto a quella attuale. Tale parametro è proporzionale al numero di collisioni per unità di tempo. Per questo, le risorse computazionali necessarie a tutti i livelli della ricostruzione cresceranno notevolmente. Dunque, la collaborazione CMS ha cominciato già da alcuni anni ad esplorare le possibilità offerte dal calcolo eterogeneo, ovvero la pratica di distribuire la computazione tra CPU e altri acceleratori dedicati, come ad esempio schede grafiche (GPU). Una delle difficoltà di questo approccio è la necessità di scrivere, validare e mantenere codice diverso per ogni dispositivo su cui dovrà essere eseguito. Questa tesi presenta la possibilità di usare SYCL per tradurre codice per la ricostruzione di eventi in modo che sia eseguibile ed efficiente su diversi dispositivi senza modifiche sostanziali. SYCL è un livello di astrazione per il calcolo eterogeneo, che rispetta lo standard ISO C++. Questo studio si concentra sul porting di un algoritmo di clustering dei depositi di energia calorimetrici, CLUE, usando oneAPI, l'implementazione SYCL supportata da Intel. Inizialmente, è stato tradotto l'algoritmo nella sua versione standalone, principalmente per prendere familiarità con SYCL e per la comodità di confronto delle performance con le versioni già esistenti. In questo caso, le prestazioni sono molto simili a quelle di codice CUDA nativo, a parità di hardware. Per validare la fisica, l'algoritmo è stato integrato all'interno di una versione ridotta del framework usato da CMS per la ricostruzione. I risultati fisici sono identici alle altre implementazioni mentre, dal punto di vista delle prestazioni computazionali, in alcuni casi, SYCL produce codice più veloce di altri livelli di astrazione adottati da CMS, presentandosi dunque come una possibilità interessante per il futuro del calcolo eterogeneo nella fisica delle alte energie.

ADAM: A Decentralized Parallel Computer Architecture Featuring Fast Thread and Data Migration and a Uniform Hardware Abstraction

The furious pace of Moore's Law is driving computer architecture into a realm where the the speed of light is the dominant factor in system latencies. The number of clock cycles to span a chip are increasing, while the number of bits that can be accessed within a clock cycle is decreasing. Hence, it is becoming more difficult to hide latency. One alternative solution is to reduce latency by migrating threads and data, but the overhead of existing implementations has previously made migration an unserviceable solution so far. I present an architecture, implementation, and mechanisms that reduces the overhead of migration to the point where migration is a viable supplement to other latency hiding mechanisms, such as multithreading. The architecture is abstract, and presents programmers with a simple, uniform fine-grained multithreaded parallel programming model with implicit memory management. In other words, the spatial nature and implementation details (such as the number of processors) of a parallel machine are entirely hidden from the programmer. Compiler writers are encouraged to devise programming languages for the machine that guide a programmer to express their ideas in terms of objects, since objects exhibit an inherent physical locality of data and code. The machine implementation can then leverage this locality to automatically distribute data and threads across the physical machine by using a set of high performance migration mechanisms. An implementation of this architecture could migrate a null thread in 66 cycles -- over a factor of 1000 improvement over previous work. Performance also scales well; the time required to move a typical thread is only 4 to 5 times that of a null thread. Data migration performance is similar, and scales linearly with data block size. Since the performance of the migration mechanism is on par with that of an L2 cache, the implementation simulated in my work has no data caches and relies instead on multithreading and the migration mechanism to hide and reduce access latencies.

Estensione dell'Abstraction Layer di DTNperf alle API di IBR-DTN

Sono dette “challenged networks” quelle reti in cui lunghi ritardi, frequenti partizionamenti e interruzioni, elevati tassi di errore e di perdita non consentono l’impiego dei classici protocolli di comunicazione di Internet, in particolare il TCP/IP. Il Delay-/Disruption-Tolerant Networking (DTN) è una soluzione per il trasferimento di dati attraverso queste reti. L’architettura DTN prevede l’introduzione, sopra il livello di trasporto, del cosiddetto “bundle layer”, che si occupa di veicolare messaggi, o bundle, secondo l’approccio store-and-forward: ogni nodo DTN conserva persistentemente un bundle finché non si presenta l’opportunità di inoltrarlo al nodo successivo verso la destinazione. Il protocollo impiegato nel bundle layer è il Bundle Protocol, le cui principali implementazioni sono tre: DTN2, l’implementazione di riferimento; ION, sviluppata da NASA-JPL e più orientata alle comunicazioni spaziali; IBR-DTN, rivolta soprattutto a dispositivi embedded. Ciascuna di esse offre API che consentono la scrittura di applicazioni in grado di inviare e ricevere bundle. DTNperf è uno strumento progettato per la valutazione delle prestazioni in ambito DTN. La più recente iterazione, DTNperf_3, è compatibile sia con DTN2 che con ION nella stessa versione del programma, grazie all’introduzione di un “Abstraction Layer” che fornisce un’unica interfaccia per l’interazione con le diverse implementazioni del Bundle Protocol e che solo internamente si occupa di invocare le API specifiche dell’implementazione attiva. Obiettivo della tesi è estendere l’Abstraction Layer affinché supporti anche IBR-DTN, cosicché DTNperf_3 possa essere impiegato indifferentemente su DTN2, ION e IBR DTN. Il lavoro sarà ripartito su tre fasi: nella prima esploreremo IBR DTN e le sue API; nella seconda procederemo all’effettiva estensione dell’Abstraction Layer; nella terza verificheremo il funzionamento di DTNperf a seguito delle modifiche, sia in ambiente esclusivamente IBR-DTN, sia ibrido.

RTOS Framework for Real-Time Control System

This thesis is done as a complementary part for the active magnet bearing (AMB) control software development project in Lappeenranta University of Technology. The main focus of the thesis is to examine an idea of a real-time operating system (RTOS) framework that operates in a dedicated digital signal processor (DSP) environment. General use real-time operating systems do not necessarily provide sufficient platform for periodic control algorithm utilisation. In addition, application program interfaces found in real-time operating systems are commonly non-existent or provided as chip-support libraries, thus hindering platform independent software development. Hence, two divergent real-time operating systems and additional periodic extension software with the framework design are examined to find solutions for the research problems. The research is discharged by; tracing the selected real-time operating system, formulating requirements for the system, and designing the real-time operating system framework (OSFW). The OSFW is formed by programming the framework and conjoining the outcome with the RTOS and the periodic extension. The system is tested and functionality of the software is evaluated in theoretical context of the Rate Monotonic Scheduling (RMS) theory. The performance of the OSFW and substance of the approach are discussed in contrast to the research theme. The findings of the thesis demonstrates that the forged real-time operating system framework is a viable groundwork solution for periodic control applications.

Design and implementation of a behaviorally typed programming system for web services

Dissertação apresentada na Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa para a obtenção do Grau de Mestre em Engenharia Informática.

Multithreading RTOS processor design

Tese de Doutoramento Plano Doutoral em Engenharia Eletrónica e de Computadores.

Memory resource balancing for virtualized computing

Virtualization has become a common abstraction layer in modern data centers. By multiplexing hardware resources into multiple virtual machines (VMs) and thus enabling several operating systems to run on the same physical platform simultaneously, it can effectively reduce power consumption and building size or improve security by isolating VMs. In a virtualized system, memory resource management plays a critical role in achieving high resource utilization and performance. Insufficient memory allocation to a VM will degrade its performance dramatically. On the contrary, over-allocation causes waste of memory resources. Meanwhile, a VM’s memory demand may vary significantly. As a result, effective memory resource management calls for a dynamic memory balancer, which, ideally, can adjust memory allocation in a timely manner for each VM based on their current memory demand and thus achieve the best memory utilization and the optimal overall performance. In order to estimate the memory demand of each VM and to arbitrate possible memory resource contention, a widely proposed approach is to construct an LRU-based miss ratio curve (MRC), which provides not only the current working set size (WSS) but also the correlation between performance and the target memory allocation size. Unfortunately, the cost of constructing an MRC is nontrivial. In this dissertation, we first present a low overhead LRU-based memory demand tracking scheme, which includes three orthogonal optimizations: AVL-based LRU organization, dynamic hot set sizing and intermittent memory tracking. Our evaluation results show that, for the whole SPEC CPU 2006 benchmark suite, after applying the three optimizing techniques, the mean overhead of MRC construction is lowered from 173% to only 2%. Based on current WSS, we then predict its trend in the near future and take different strategies for different prediction results. When there is a sufficient amount of physical memory on the host, it locally balances its memory resource for the VMs. Once the local memory resource is insufficient and the memory pressure is predicted to sustain for a sufficiently long time, a relatively expensive solution, VM live migration, is used to move one or more VMs from the hot host to other host(s). Finally, for transient memory pressure, a remote cache is used to alleviate the temporary performance penalty. Our experimental results show that this design achieves 49% center-wide speedup.

Multi-GPU support on the marrow algorithmic skeleton framework

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

Concurrent : Easy to use Python computing environment

Concurrent aims to be a different type of task distribution system compared to what MPI like system do. It adds a simple but powerful application abstraction layer to distribute the logic of an entire application onto a swarm of clusters holding similarities with volunteer computing systems. Traditional task distributed systems will just perform simple tasks onto the distributed system and wait for results. Concurrent goes one step further by letting the tasks and the application decide what to do. The programming paradigm is then totally async without any waits for results and based on notifications once a computation has been performed.

A tool to simplify the management of homogeneous and heterogeneous grids

To simplify computer management, several system administrators are adopting advanced techniques to manage software configuration on grids, but the tight coupling between hardware and software makes every PC an individual managed entity, lowering the scalability and increasing the costs to manage hundreds or thousands of PCs. This paper discusses the feasibility of a distributed virtual machine environment, named Flexlab: a new approach for computer management that combines virtualization and distributed system architectures as the basis of a management system. Flexlab is able to extend the coverage of a computer management solution beyond client operating system limitations and also offers a convenient hardware abstraction, decoupling software and hardware, simplifying computer management. The results obtained in this work indicate that FlexLab is able to overcome the limitations imposed by the coupling between software and hardware, simplifying the management of homogeneous and heterogeneous grids. © 2009 IEEE.

TORP: An open standard framework for humanoid robots

Humanoid robots are an extremely complex interdisciplinary research field. Particularly, the development of high size humanoid robots usually requires joint efforts and skills from groups that are in many different research centers around the world. However, there are serious constraints in this kind of collaborative development. Some efforts have been made in order to propose new software frameworks that can allow distributed development with also some degree of hardware abstraction, allowing software reuse in successive projects. However, computation represents only one of the dimensions in robotics tasks, and the need for reuse and exchange of full robot modules between groups are growing. Large advances could be reached if physical parts of a robot could be reused in a different robot constructed with other technologies by other researcher or group. This paper proposes a new robot framework, from now on called TORP (The Open Robot Project), that aims to provide a standard architecture in all dimensions (electrical, mechanical and computational) for this collaborative development. This methodology also represents an open project that is fully shared. In this paper, the first robot constructed following the TORP specification set is presented as well as the advances proposed for its improvement. © 2010 IEEE.

Middleware de comunicação entre objetos distribuídos para gerenciamento de computadores baseado em redes sem fio (WSE-OS)

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Analysis, monitoring, and management of quality of experience in video delivery services over IP

Esta tesis estudia la monitorización y gestión de la Calidad de Experiencia (QoE) en los servicios de distribución de vídeo sobre IP. Aborda el problema de cómo prevenir, detectar, medir y reaccionar a las degradaciones de la QoE desde la perspectiva de un proveedor de servicios: la solución debe ser escalable para una red IP extensa que entregue flujos individuales a miles de usuarios simultáneamente. La solución de monitorización propuesta se ha denominado QuEM(Qualitative Experience Monitoring, o Monitorización Cualitativa de la Experiencia). Se basa en la detección de las degradaciones de la calidad de servicio de red (pérdidas de paquetes, disminuciones abruptas del ancho de banda...) e inferir de cada una una descripción cualitativa de su efecto en la Calidad de Experiencia percibida (silencios, defectos en el vídeo...). Este análisis se apoya en la información de transporte y de la capa de abstracción de red de los flujos codificados, y permite caracterizar los defectos más relevantes que se observan en este tipo de servicios: congelaciones, efecto de “cuadros”, silencios, pérdida de calidad del vídeo, retardos e interrupciones en el servicio. Los resultados se han validado mediante pruebas de calidad subjetiva. La metodología usada en esas pruebas se ha desarrollado a su vez para imitar lo más posible las condiciones de visualización de un usuario de este tipo de servicios: los defectos que se evalúan se introducen de forma aleatoria en medio de una secuencia de vídeo continua. Se han propuesto también algunas aplicaciones basadas en la solución de monitorización: un sistema de protección desigual frente a errores que ofrece más protección a las partes del vídeo más sensibles a pérdidas, una solución para minimizar el impacto de la interrupción de la descarga de segmentos de Streaming Adaptativo sobre HTTP, y un sistema de cifrado selectivo que encripta únicamente las partes del vídeo más sensibles. También se ha presentado una solución de cambio rápido de canal, así como el análisis de la aplicabilidad de los resultados anteriores a un escenario de vídeo en 3D. ABSTRACT This thesis proposes a comprehensive approach to the monitoring and management of Quality of Experience (QoE) in multimedia delivery services over IP. It addresses the problem of preventing, detecting, measuring, and reacting to QoE degradations, under the constraints of a service provider: the solution must scale for a wide IP network delivering individual media streams to thousands of users. The solution proposed for the monitoring is called QuEM (Qualitative Experience Monitoring). It is based on the detection of degradations in the network Quality of Service (packet losses, bandwidth drops...) and the mapping of each degradation event to a qualitative description of its effect in the perceived Quality of Experience (audio mutes, video artifacts...). This mapping is based on the analysis of the transport and Network Abstraction Layer information of the coded stream, and allows a good characterization of the most relevant defects that exist in this kind of services: screen freezing, macroblocking, audio mutes, video quality drops, delay issues, and service outages. The results have been validated by subjective quality assessment tests. The methodology used for those test has also been designed to mimic as much as possible the conditions of a real user of those services: the impairments to evaluate are introduced randomly in the middle of a continuous video stream. Based on the monitoring solution, several applications have been proposed as well: an unequal error protection system which provides higher protection to the parts of the stream which are more critical for the QoE, a solution which applies the same principles to minimize the impact of incomplete segment downloads in HTTP Adaptive Streaming, and a selective scrambling algorithm which ciphers only the most sensitive parts of the media stream. A fast channel change application is also presented, as well as a discussion about how to apply the previous results and concepts in a 3D video scenario.

Virtualización de laboratorios

Para empezar, se ha hecho un análisis de las diferentes posibilidades que se podían implementar para poder conseguir el objetivo del trabajo. El resultado final debe ser, disponer de máquinas para que el sistema operativo fuese independiente del hardware que se tiene instalado en él . Para ello, se decide montar un sistema operativo de base en todos los equipos del laboratorio, que tenga las necesidades mínimas que se necesitan, las cuales son una interfaz gráfica y conexión de red. Hay que intentar reducir el consumo de recursos al máximo con este sistema operativo mínimo para que el rendimiento de las máquinas sea lo más fluido posible para los usuarios. El sistema elegido fue Linux con su distribución Ubuntu [ubu, http] con los módulos mínimos que permita funcionar el software necesario. Una vez se instala el sistema operativo anfitrión, se instala el escritorio Xfce [ubu2, http], que es el más ligero de Ubuntu, pero que proporciona buen rendimiento. Después, se procedió a instalar un software de virtualización en cada equipo. En este caso se decidió, por las buenas prestaciones que ofrecía, que fuera VirtualBox [vir2,http] de Oracle. Sobre éste software se crean tantas máquinas virtuales (con sistema operativo Windows) como asignaturas diferentes se cursan en el laboratorio donde se trabaje. Con esto, se consigue que al arrancar el programa los alumnos pudieran escoger qué máquina arrancar y lo que es más importante, se permite realizar cualquier cambio en el hardware (exceptuando el disco duro porque borraría todo lo que se tuviera guardado). Además de no tener que volver a reinstalar el sistema operativo nuevamente, se consigue la abstracción del software y hardware. También se decide que, para tener un respaldo de las máquinas virtuales que se tengan creadas en VirtualBox, se utiliza un servidor NAS. Uno de los motivos de utilizar dicho servidor fue por aprovechar una infraestructura ya creada. Un servidor NAS da la posibilidad de recuperar cualquier archivo (máquina virtual) cuando haga falta porque haya alguna máquina virtual corrupta en algún equipo, o en varios. Este tipo de servidor tiene la gran ventaja de ser multicast, es decir, permite solicitudes simultáneas. ABSTRACT For starters, there has been an analysis of the different possibilities that could be implemented to achieve the objective of the work. This objective was to have machines for the operating system to be independent of the hardware we have installed on it. Therefore, we decided to create an operating system based on all computers in the laboratory, taking the minimum needs we need. This is a graphical interface and network connection. We must try to reduce the consumption of resources to the maximum for the performance of the machines is as fluid as possible for users. The system was chosen with its Ubuntu Linux distribution with minimum modules that allow us to run software that is necessary for us. Once the base is installed, we install the Xfce desktop, which is the lightest of Ubuntu, but which provided good performance. Then we proceeded to install a virtualization software on each computer. In this case we decided, for good performance that gave us, it was Oracle VirtualBox. About this software create many virtual machines (Windows operating system) as different subjects are studied in the laboratory where we are. With that, we got it at program startup students could choose which machine start and what is more important, allowed us to make any changes to the hardware (except the hard drive because it would erase all we have). Besides not having to reinstall the operating system again, we get the software and hardware abstraction. We also decided that in order to have a backup of our virtual machines that we created in VirtualBox, we use a NAS server. One reason to use that server was to leverage their existing network infrastructure. A NAS server gives us the ability to retrieve any file (image) when we do need because there is some corrupt virtual machine in a team, or several. This is possible because this type of server allows multicast connection.